{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Link prediction example: GCN on the Cora citation dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we use our implementation of the [GCN](https://arxiv.org/abs/1609.02907) algorithm to build a model that predicts citation links in the Cora dataset (see below). The problem is treated as a supervised link prediction problem on a homogeneous citation network with nodes representing papers (with attributes such as binary keyword indicators and categorical subject) and links corresponding to paper-paper citations. \n",
    "\n",
    "To address this problem, we build a model with the following architecture. First we build a two-layer GCN model that takes labeled node pairs (`citing-paper` -> `cited-paper`)  corresponding to possible citation links, and outputs a pair of node embeddings for the `citing-paper` and `cited-paper` nodes of the pair. These embeddings are then fed into a link classification layer, which first applies a binary operator to those node embeddings (e.g., concatenating them) to construct the embedding of the potential link. Thus obtained link embeddings are passed through the dense link classification layer to obtain link predictions - probability for these candidate links to actually exist in the network. The entire model is trained end-to-end by minimizing the loss function of choice (e.g., binary cross-entropy between predicted link probabilities and true link labels, with true/false citation links having labels 1/0) using stochastic gradient descent (SGD) updates of the model parameters, with minibatches of 'training' links fed into the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "import pandas as pd\n",
    "import os\n",
    "\n",
    "import stellargraph as sg\n",
    "from stellargraph.data import EdgeSplitter\n",
    "from stellargraph.mapper import FullBatchLinkGenerator\n",
    "from stellargraph.layer import GCN, LinkEmbedding\n",
    "\n",
    "\n",
    "from tensorflow import keras\n",
    "from sklearn import preprocessing, feature_extraction, model_selection\n",
    "\n",
    "from stellargraph import globalvar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the CORA network data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Downloading the CORA dataset:**\n",
    "    \n",
    "The dataset used in this demo can be downloaded from https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz\n",
    "\n",
    "The following is the description of the dataset:\n",
    "> The Cora dataset consists of 2708 scientific publications classified into one of seven classes.\n",
    "> The citation network consists of 5429 links. Each publication in the dataset is described by a\n",
    "> 0/1-valued word vector indicating the absence/presence of the corresponding word from the dictionary.\n",
    "> The dictionary consists of 1433 unique words. The README file in the dataset provides more details.\n",
    "\n",
    "Download and unzip the cora.tgz file to a location on your computer and set the `data_dir` variable to\n",
    "point to the location of the dataset (the directory containing \"cora.cites\" and \"cora.content\")."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dir = os.path.expanduser(\"~/data/cora\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the graph from edgelist (in `cited-paper`,`citing-paper` order)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "edgelist = pd.read_csv(\n",
    "    os.path.join(data_dir, \"cora.cites\"),\n",
    "    sep=\"\\t\",\n",
    "    header=None,\n",
    "    names=[\"target\", \"source\"],\n",
    ")\n",
    "edgelist[\"label\"] = \"cites\"  # set the edge type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "G = nx.from_pandas_edgelist(edgelist, edge_attr=\"label\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the features and subject for the nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_names = [\"w_{}\".format(ii) for ii in range(1433)]\n",
    "column_names = feature_names + [\"subject\"]\n",
    "node_data = pd.read_csv(\n",
    "    os.path.join(data_dir, \"cora.content\"), sep=\"\\t\", header=None, names=column_names\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define a set of node features that will be used by the model as the difference between the set of all node features and a list of user-defined node attributes to ignore:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "ignore_attr = []\n",
    "feature_names = sorted(set(column_names) - set(ignore_attr))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to convert node features that will be used by the model to numeric values. Note that all node features in the Cora dataset, except the categorical \"subject\" feature, are already numeric, and don't require the conversion."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "if \"subject\" in feature_names:\n",
    "    # Convert node features to numeric vectors\n",
    "    feature_encoding = feature_extraction.DictVectorizer(sparse=False)\n",
    "    node_features = feature_encoding.fit_transform(\n",
    "        node_data[feature_names].to_dict(\"records\")\n",
    "    )\n",
    "else:  # node features are already numeric, no further conversion is needed\n",
    "    node_features = node_data[feature_names].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add node data to G:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "for nid, f in zip(node_data.index, node_features):\n",
    "    G.nodes[nid][globalvar.TYPE_ATTR_NAME] = \"paper\"  # specify node type\n",
    "    G.nodes[nid][\"feature\"] = f"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We aim to train a link prediction model, hence we need to prepare the train and test sets of links and the corresponding graphs with those links removed.\n",
    "\n",
    "We are going to split our input graph into a train and test graphs using the EdgeSplitter class in `stellargraph.data`. We will use the train graph for training the model (a binary classifier that, given two nodes, predicts whether a link between these two nodes should exist or not) and the test graph for evaluating the model's performance on hold out data.\n",
    "Each of these graphs will have the same number of nodes as the input graph, but the number of links will differ (be reduced) as some of the links will be removed during each split and used as the positive samples for training/testing the link prediction classifier."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the original graph G, extract a randomly sampled subset of test edges (true and false citation links) and the reduced graph G_test with the positive test edges removed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "** Sampled 527 positive and 527 negative edges. **\n"
     ]
    }
   ],
   "source": [
    "# Define an edge splitter on the original graph G:\n",
    "edge_splitter_test = EdgeSplitter(G)\n",
    "\n",
    "# Randomly sample a fraction p=0.1 of all positive links, and same number of negative links, from G, and obtain the\n",
    "# reduced graph G_test with the sampled links removed:\n",
    "G_test, edge_ids_test, edge_labels_test = edge_splitter_test.train_test_split(\n",
    "    p=0.1, method=\"global\", keep_connected=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reduced graph G_test, together with the test ground truth set of links (edge_ids_test, edge_labels_test), will be used for testing the model.\n",
    "\n",
    "Now repeat this procedure to obtain the training data for the model. From the reduced graph G_test, extract a randomly sampled subset of train edges (true and false citation links) and the reduced graph G_train with the positive train edges removed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "** Sampled 475 positive and 475 negative edges. **\n"
     ]
    }
   ],
   "source": [
    "# Define an edge splitter on the reduced graph G_test:\n",
    "edge_splitter_train = EdgeSplitter(G_test)\n",
    "\n",
    "# Randomly sample a fraction p=0.1 of all positive links, and same number of negative links, from G_test, and obtain the\n",
    "# reduced graph G_train with the sampled links removed:\n",
    "G_train, edge_ids_train, edge_labels_train = edge_splitter_train.train_test_split(\n",
    "    p=0.1, method=\"global\", keep_connected=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "G_train, together with the train ground truth set of links (edge_ids_train, edge_labels_train), will be used for training the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert G_train and G_test to StellarGraph objects (undirected, as required by GCN) for ML:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "G_train = sg.StellarGraph(G_train, node_features=\"feature\")\n",
    "G_test = sg.StellarGraph(G_test, node_features=\"feature\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating the GCN link model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we create the link generators for the train and test link examples to the model. The link generators take the pairs of nodes (`citing-paper`, `cited-paper`) that are given in the `.flow` method to the Keras model, together with the corresponding binary labels indicating whether those pairs represent true or false links.\n",
    "\n",
    "The number of epochs for training the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For training we create a generator on the `G_train` graph, and make an iterator over the training links using the generator's `flow()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using GCN (local pooling) filters...\n"
     ]
    }
   ],
   "source": [
    "train_gen = FullBatchLinkGenerator(G_train, method=\"gcn\")\n",
    "train_flow = train_gen.flow(edge_ids_train, edge_labels_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using GCN (local pooling) filters...\n"
     ]
    }
   ],
   "source": [
    "test_gen = FullBatchLinkGenerator(G_test, method=\"gcn\")\n",
    "test_flow = train_gen.flow(edge_ids_test, edge_labels_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can specify our machine learning model, we need a few more parameters for this:\n",
    "\n",
    " * the `layer_sizes` is a list of hidden feature sizes of each layer in the model. In this example we use two GCN layers with 16-dimensional hidden node features at each layer.\n",
    " * `activations` is a list of activations applied to each layer's output\n",
    " * `dropout=0.3` specifies a 30% dropout at each layer. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create a GCN model as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "gcn = GCN(\n",
    "    layer_sizes=[16, 16], activations=[\"relu\", \"relu\"], generator=train_gen, dropout=0.3\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create a Keras model we now expose the input and output tensors of the GCN model for link prediction, via the `GCN.build` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_inp, x_out = gcn.build()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Final link classification layer that takes a pair of node embeddings produced by the GCN model, applies a binary operator to them to produce the corresponding link embedding ('ip' for inner product; other options for the binary operator can be seen by running a cell with `?LinkEmbedding` in it), and passes it through a dense layer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "prediction = LinkEmbedding(activation=\"relu\", method=\"ip\")(x_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The predictions need to be reshaped from `(X, 1)` to `(X,)` to match the shape of the targets we have supplied above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "prediction = keras.layers.Reshape((-1,))(prediction)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Stack the GCN and prediction layers into a Keras model, and specify the loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = keras.Model(inputs=x_inp, outputs=prediction)\n",
    "\n",
    "model.compile(\n",
    "    optimizer=keras.optimizers.Adam(lr=0.01),\n",
    "    loss=keras.losses.binary_crossentropy,\n",
    "    metrics=[\"acc\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the initial (untrained) model on the train and test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Set Metrics of the initial (untrained) model:\n",
      "\tloss: 1.9300\n",
      "\tacc: 0.5000\n",
      "\n",
      "Test Set Metrics of the initial (untrained) model:\n",
      "\tloss: 1.9193\n",
      "\tacc: 0.5000\n"
     ]
    }
   ],
   "source": [
    "init_train_metrics = model.evaluate_generator(train_flow)\n",
    "init_test_metrics = model.evaluate_generator(test_flow)\n",
    "\n",
    "print(\"\\nTrain Set Metrics of the initial (untrained) model:\")\n",
    "for name, val in zip(model.metrics_names, init_train_metrics):\n",
    "    print(\"\\t{}: {:0.4f}\".format(name, val))\n",
    "\n",
    "print(\"\\nTest Set Metrics of the initial (untrained) model:\")\n",
    "for name, val in zip(model.metrics_names, init_test_metrics):\n",
    "    print(\"\\t{}: {:0.4f}\".format(name, val))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Train the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/50\n",
      "1/1 - 0s - loss: 1.8187 - acc: 0.5000 - val_loss: 1.8040 - val_acc: 0.5455\n",
      "Epoch 2/50\n",
      "1/1 - 0s - loss: 2.2420 - acc: 0.5442 - val_loss: 0.7943 - val_acc: 0.6157\n",
      "Epoch 3/50\n",
      "1/1 - 0s - loss: 0.9887 - acc: 0.6074 - val_loss: 0.7350 - val_acc: 0.5588\n",
      "Epoch 4/50\n",
      "1/1 - 0s - loss: 0.7206 - acc: 0.5947 - val_loss: 0.7700 - val_acc: 0.5427\n",
      "Epoch 5/50\n",
      "1/1 - 0s - loss: 0.7529 - acc: 0.5653 - val_loss: 0.7496 - val_acc: 0.5560\n",
      "Epoch 6/50\n",
      "1/1 - 0s - loss: 0.7073 - acc: 0.5895 - val_loss: 0.7048 - val_acc: 0.5797\n",
      "Epoch 7/50\n",
      "1/1 - 0s - loss: 0.6452 - acc: 0.6432 - val_loss: 0.6886 - val_acc: 0.6347\n",
      "Epoch 8/50\n",
      "1/1 - 0s - loss: 0.6463 - acc: 0.6726 - val_loss: 0.6864 - val_acc: 0.6110\n",
      "Epoch 9/50\n",
      "1/1 - 0s - loss: 0.6265 - acc: 0.6789 - val_loss: 0.6917 - val_acc: 0.6091\n",
      "Epoch 10/50\n",
      "1/1 - 0s - loss: 0.6234 - acc: 0.6800 - val_loss: 0.6834 - val_acc: 0.6139\n",
      "Epoch 11/50\n",
      "1/1 - 0s - loss: 0.6345 - acc: 0.6811 - val_loss: 0.6869 - val_acc: 0.6195\n",
      "Epoch 12/50\n",
      "1/1 - 0s - loss: 0.5710 - acc: 0.6947 - val_loss: 0.6801 - val_acc: 0.6252\n",
      "Epoch 13/50\n",
      "1/1 - 0s - loss: 0.5798 - acc: 0.7126 - val_loss: 0.6956 - val_acc: 0.6395\n",
      "Epoch 14/50\n",
      "1/1 - 0s - loss: 0.5634 - acc: 0.7221 - val_loss: 0.6921 - val_acc: 0.6471\n",
      "Epoch 15/50\n",
      "1/1 - 0s - loss: 0.5778 - acc: 0.7358 - val_loss: 0.6810 - val_acc: 0.6584\n",
      "Epoch 16/50\n",
      "1/1 - 0s - loss: 0.5236 - acc: 0.7495 - val_loss: 0.6730 - val_acc: 0.6651\n",
      "Epoch 17/50\n",
      "1/1 - 0s - loss: 0.5660 - acc: 0.7295 - val_loss: 0.6649 - val_acc: 0.6717\n",
      "Epoch 18/50\n",
      "1/1 - 0s - loss: 0.4976 - acc: 0.7432 - val_loss: 0.6579 - val_acc: 0.6850\n",
      "Epoch 19/50\n",
      "1/1 - 0s - loss: 0.5007 - acc: 0.7579 - val_loss: 0.6629 - val_acc: 0.6917\n",
      "Epoch 20/50\n",
      "1/1 - 0s - loss: 0.5152 - acc: 0.7695 - val_loss: 0.6337 - val_acc: 0.6973\n",
      "Epoch 21/50\n",
      "1/1 - 0s - loss: 0.4662 - acc: 0.7768 - val_loss: 0.6275 - val_acc: 0.7040\n",
      "Epoch 22/50\n",
      "1/1 - 0s - loss: 0.4722 - acc: 0.7779 - val_loss: 0.6332 - val_acc: 0.7049\n",
      "Epoch 23/50\n",
      "1/1 - 0s - loss: 0.4502 - acc: 0.7989 - val_loss: 0.6462 - val_acc: 0.7030\n",
      "Epoch 24/50\n",
      "1/1 - 0s - loss: 0.4438 - acc: 0.8095 - val_loss: 0.6420 - val_acc: 0.7068\n",
      "Epoch 25/50\n",
      "1/1 - 0s - loss: 0.4315 - acc: 0.8158 - val_loss: 0.6630 - val_acc: 0.7059\n",
      "Epoch 26/50\n",
      "1/1 - 0s - loss: 0.4399 - acc: 0.8189 - val_loss: 0.6739 - val_acc: 0.7116\n",
      "Epoch 27/50\n",
      "1/1 - 0s - loss: 0.4640 - acc: 0.8284 - val_loss: 0.6698 - val_acc: 0.7173\n",
      "Epoch 28/50\n",
      "1/1 - 0s - loss: 0.4515 - acc: 0.8295 - val_loss: 0.6667 - val_acc: 0.7239\n",
      "Epoch 29/50\n",
      "1/1 - 0s - loss: 0.3956 - acc: 0.8389 - val_loss: 0.6643 - val_acc: 0.7201\n",
      "Epoch 30/50\n",
      "1/1 - 0s - loss: 0.3846 - acc: 0.8337 - val_loss: 0.6616 - val_acc: 0.7182\n",
      "Epoch 31/50\n",
      "1/1 - 0s - loss: 0.3953 - acc: 0.8589 - val_loss: 0.6590 - val_acc: 0.7201\n",
      "Epoch 32/50\n",
      "1/1 - 0s - loss: 0.4024 - acc: 0.8526 - val_loss: 0.6576 - val_acc: 0.7211\n",
      "Epoch 33/50\n",
      "1/1 - 0s - loss: 0.3868 - acc: 0.8484 - val_loss: 0.6681 - val_acc: 0.7230\n",
      "Epoch 34/50\n",
      "1/1 - 0s - loss: 0.3695 - acc: 0.8558 - val_loss: 0.6724 - val_acc: 0.7249\n",
      "Epoch 35/50\n",
      "1/1 - 0s - loss: 0.3695 - acc: 0.8684 - val_loss: 0.7002 - val_acc: 0.7258\n",
      "Epoch 36/50\n",
      "1/1 - 0s - loss: 0.3540 - acc: 0.8779 - val_loss: 0.7207 - val_acc: 0.7249\n",
      "Epoch 37/50\n",
      "1/1 - 0s - loss: 0.3643 - acc: 0.8705 - val_loss: 0.7329 - val_acc: 0.7268\n",
      "Epoch 38/50\n",
      "1/1 - 0s - loss: 0.3733 - acc: 0.8558 - val_loss: 0.7207 - val_acc: 0.7277\n",
      "Epoch 39/50\n",
      "1/1 - 0s - loss: 0.3493 - acc: 0.8863 - val_loss: 0.7190 - val_acc: 0.7230\n",
      "Epoch 40/50\n",
      "1/1 - 0s - loss: 0.3254 - acc: 0.8832 - val_loss: 0.7206 - val_acc: 0.7287\n",
      "Epoch 41/50\n",
      "1/1 - 0s - loss: 0.3598 - acc: 0.8779 - val_loss: 0.7535 - val_acc: 0.7334\n",
      "Epoch 42/50\n",
      "1/1 - 0s - loss: 0.3260 - acc: 0.8768 - val_loss: 0.7531 - val_acc: 0.7343\n",
      "Epoch 43/50\n",
      "1/1 - 0s - loss: 0.3212 - acc: 0.8842 - val_loss: 0.7523 - val_acc: 0.7372\n",
      "Epoch 44/50\n",
      "1/1 - 0s - loss: 0.2966 - acc: 0.8863 - val_loss: 0.7540 - val_acc: 0.7372\n",
      "Epoch 45/50\n",
      "1/1 - 0s - loss: 0.3057 - acc: 0.8874 - val_loss: 0.7369 - val_acc: 0.7343\n",
      "Epoch 46/50\n",
      "1/1 - 0s - loss: 0.3234 - acc: 0.8916 - val_loss: 0.7105 - val_acc: 0.7353\n",
      "Epoch 47/50\n",
      "1/1 - 0s - loss: 0.2769 - acc: 0.8916 - val_loss: 0.6903 - val_acc: 0.7353\n",
      "Epoch 48/50\n",
      "1/1 - 0s - loss: 0.2989 - acc: 0.8842 - val_loss: 0.6981 - val_acc: 0.7372\n",
      "Epoch 49/50\n",
      "1/1 - 0s - loss: 0.2684 - acc: 0.9000 - val_loss: 0.7014 - val_acc: 0.7410\n",
      "Epoch 50/50\n",
      "1/1 - 0s - loss: 0.2917 - acc: 0.8895 - val_loss: 0.7457 - val_acc: 0.7429\n"
     ]
    }
   ],
   "source": [
    "history = model.fit_generator(\n",
    "    train_flow, epochs=epochs, validation_data=test_flow, verbose=2, shuffle=False\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the training history:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "def plot_history(history):\n",
    "    metrics = sorted(history.history.keys())\n",
    "    metrics = metrics[:len(metrics)//2]\n",
    "    \n",
    "    f,axs = plt.subplots(1, len(metrics), figsize=(12,4))\n",
    "\n",
    "    for m,ax in zip(metrics,axs):\n",
    "        # summarize history for metric m\n",
    "        ax.plot(history.history[m])\n",
    "        ax.plot(history.history['val_' + m])\n",
    "        ax.set_title(m)\n",
    "        ax.set_ylabel(m)\n",
    "        ax.set_xlabel('epoch')\n",
    "        ax.legend(['train', 'test'], loc='best')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtcAAAEWCAYAAACt0rvRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3yV9fn/8deVdbIDJAGEEAjIBkEIiOKAqgjWiqNVcLYqqFV/bbVWtFarHfLtUOuuKCoOXDioooAVirggTNkjAgkgZJAQssf1++M+gUMWIeSMJNfz8cgj59zjnCul3nnncz739RFVxRhjjDHGGHPigvxdgDHGGGOMMa2FhWtjjDHGGGOaiYVrY4wxxhhjmomFa2OMMcYYY5qJhWtjjDHGGGOaiYVrY4wxxhhjmomFa2OMMca0CCKyQ0TO83cdxjTEwrUxxhhjjDHNxMK1McYYY4wxzcTCtTGAiEwTke0iUiAiG0TkUo99U0Rko8e+Ye7t3UTkPRHJEpEcEXnKfz+BMca0HSLiEpHHRWSP++txEXG59yWIyEcikiciuSLyhYgEuffdIyK73dfzzSJyrn9/EtMahfi7AGMCxHbgLOAH4GfAayJyMnAm8EfgEiAN6AWUi0gw8BHwOXAtUAmk+r5sY4xpk34PjAKGAgp8CNwP/AG4C8gEEt3HjgJURPoCtwMjVHWPiPQAgn1btmkLbOTaGEBV31HVPapapapvAVuBkcBNwN9Udbk6tqnqTve+LsDdqlqoqiWqutSPP4IxxrQlVwMPq+p+Vc0CHsIZ6AAoB04Cuqtquap+oaqKMwjiAgaISKiq7lDV7X6p3rRqFq6NAUTkOhFZ7f4YMQ8YBCQA3XBGtWvqBuxU1Qpf1mmMMQZwBjd2ejzf6d4G8HdgG7BARNJFZBqAqm4Dfo3zaeR+EXlTRLpgTDOzcG3aPBHpDszA+bgwXlXbAesAATJwpoLUlAEki4hNrTLGGN/bA3T3eJ7s3oaqFqjqXaraE7gYuLN6brWqvqGqZ7rPVeD/fFu2aQssXBsDUTgX2SwAEfkFzsg1wAvAb0VkuDhOdofxZcBeYLqIRIlIuIiM9kfxxhjTBs0G7heRRBFJAB4AXgMQkYvc12oB8nGmg1SJSF8R+ZH7xscSoBio8lP9phWzcG3aPFXdAPwT+BrYBwwGvnTvewf4C/AGUAB8AHRQ1UrgJ8DJwC6cm2eu9HnxxhjTNv0Z5ybztcB3wEr3NoDewGfAIZzr+jOqughnvvV0IBvn5vWOwL2+Ldu0BeLM8TfGGGOMMcacKBu5NsYYY4wxpplYuDbGGGOMMaaZWLg2xhhjjDGmmVi4NsYYY4wxppm0mh69CQkJ2qNHD3+XYYwxTbJixYpsVU089pGth123jTEtVUPX7FYTrnv06EFaWpq/yzDGmCYRkZ3HPqp1seu2MaalauiabdNCjDHGGGOMaSYWro0xxhhjjGkmFq6NMcYYY4xpJq1mznVdysvLyczMpKSkxN+leF14eDhJSUmEhob6uxRjjDHGtHJtJWM1JV+16nCdmZlJTEwMPXr0QET8XY7XqCo5OTlkZmaSkpLi73KMMS2QiHQDZgGdAAWeV9V/1TjmauAeQIAC4FZVXePet8O9rRKoUNVU31VvjPG1tpCxmpqvvDotRETGi8hmEdkmItPq2N9dRP4rImtFZLGIJHnsu15Etrq/rm/K+5eUlBAfH99q/9GriQjx8fGt/q9HY4xXVQB3qeoAYBRwm4gMqHHM98A5qjoY+BPwfI39Y1V1qAVrY1q/tpCxmpqvvBauRSQYeBqYAAwAJtdxof4HMEtVTwEeBh5xn9sBeBA4DRgJPCgi7ZtYR9N+gBamrfycxhjvUNW9qrrS/bgA2Ah0rXHMV6p6wP30GyAJY0yb1RayR1N+Rm+OXI8EtqlquqqWAW8CE2scMwD43P14kcf+C4CFqprrvpAvBMZ7sVZjjGm0/QUlvLV8FxWVVf4uxStEpAdwKvBtA4fdCHzi8VyBBSKyQkSmNvDaU0UkTUTSsrKymqNcduYU8r8tzfNaxhhzorwZrrsCGR7PM6kxCgKsAS5zP74UiBGR+Eae65WLdHPLy8vjmWeeOe7zLrzwQvLy8rxQkTHmRFRUVnHLqyu4Z8533Pr6SkrKK/1dUrMSkWhgDvBrVT1YzzFjccL1PR6bz1TVYTifVt4mImfXda6qPq+qqaqampjYPAtSPvX5Nn795qpmeS1jTMsQyPnK3634fgucIyKrgHOA3Tg3wzSKNy7Sza2+f/yKiooGz5s3bx7t2rXzVlnGmCZ64r9bWbkrj4uHdGHhhn1cP3MZB0vK/V1WsxCRUJxg/bqqvlfPMacALwATVTWneruq7nZ/3w+8j/PppU/szCniUGnD11RjTOsSyPnKm+F6N9DN43mSe9thqrpHVS9T1VOB37u35TXm3JZi2rRpbN++naFDhzJixAjOOussLr74YgYMcKafX3LJJQwfPpyBAwfy/PNH7g3q0aMH2dnZ7Nixg/79+zNlyhQGDhzIuHHjKC4u9tePY0yb9m16Dk8t2sblw5J4YvKp/GvSUFbsPMDk578h+1Cpv8s7IeJMLHwR2Kiqj9ZzTDLwHnCtqm7x2B4lIjHVj4FxwDrvV+3YlVtEeaVSVtE6p+kYY2oL5HzlzVZ8y4HeIpKCE4wnAVd5HiAiCUCuqlYB9wIz3bvmA3/1uIlxnHt/kz30n/Vs2FPnJ5xNNqBLLA/+ZGCDx0yfPp1169axevVqFi9ezI9//GPWrVt3uKXLzJkz6dChA8XFxYwYMYLLL7+c+Pj4o15j69atzJ49mxkzZnDFFVcwZ84crrnmmmb9WYxpK7IPlfJtei6VqrX29e0UQ9/OMXWel1dUxq/fWk1yh0gemuj8dz9xaFdiI0K59bUV/Oy5r5l1w0i6dYj0av1eNBq4FvhORFa7t90HJAOo6nPAA0A88Iz7Jp/qlnudgPfd20KAN1T1U18UXVJeyQ8HnTv5i8sqCQvx9weyxrQ9/shYgZyvvBauVbVCRG7HCcrBwExVXS8iDwNpqjoXGAM8IiIKLAFuc5+bKyJ/wgnoAA+raq63avWlkSNHHtUr8YknnuD9998HICMjg61bt9b6x09JSWHo0KEADB8+nB07dvisXmNag5LySj7ftJ85KzJZvCWLyqrawRogSODGM1O48/y+RIQFH96uqkyb8x3Zh0p579bRRLuOXDrH9u3I6zedxi9eWs5Pn/uKV288jT6d6g7ogUxVl+L0r27omJuAm+rYng4M8VJpDco8cGSkqai8gjhsIS1j2qJAyldeXURGVecB82pse8Dj8bvAu/WcO5MjI9kn7FgjzL4SFRV1+PHixYv57LPP+Prrr4mMjGTMmDF19lJ0uVyHHwcHB9u0EGMaoKocLKkgq6CUvfnFfLLuBz5as4eDJRV0inUx5ayeTBjUmSjX0Ze/KlVe+nIHM774nvnr9/HIZYMZfXICALOXZfDp+h+478J+DE6Kq/Wew7t34O1bTue6F5dx97tr+eCXZ7SJFlWBICO36PDjwtLWdXOpMS1FIGSsQMpXrXqFxkAQExNDQUFBnfvy8/Np3749kZGRbNq0iW+++cbH1RnjHY8u2EzWoTLO6ZPA6b0SiIto3tFEVSWroJQNew+ycW8BG/ceZGdOIdmHysg6VHrU3NuI0GDGD+rMZcO6ckavBIKD6g+9j1w2mIlDuzBtzlqufuFbrkhN4orUbjz80XrO6p3ATWf2rPfcfp1jmXPrGUDb6P0aKHZ5hOviMgvXxrQVgZyvLFx7WXx8PKNHj2bQoEFERETQqVOnw/vGjx/Pc889R//+/enbty+jRo3yY6XGNI9dOUU88fk2goOE2ct2ERwkDO3WjrN7J3JO30SGJMU1OXx+m57D04u3s353PjmFZYe3d20XQc/EKHp1jCYx2kVijIsE9/ch3dodNY3jWEb1jOfTX5/N459tZcYX6bydlkl8VBj/vGIIQQ0Ec6Alz7dusTzDdWGZdQwxpq0I5HwlWsdNPS1RamqqpqWlHbVt48aN9O/f308V+V5b+3lNYHps4Rae+HwrS+4eyw8HS1iyJYslW7JYuzsfVRjUNZapZ/fiwkGdCQlu3M1nBSXlTP9kE69/u4uT4sI58+QEBnSJpf9JsfTvHEtcpHfm2a7bnc/jn23lhjN7cEavBK+8RzURWdHWlg2v67p9vKbMSuOzjftQhZd+PoKx/To2U3XGmIa0pcxR18/a0DXbRq6NMc2mqkqZszKTM09OoFuHSLp1iGREjw7cNa4vuYVlzF//AzO+SOf/zV7F39pHMOWsnvwsNYnIsPovRZ9t2Mf9H6xjf0EJN52Zwp3j+jR4fHMa1DWOF65vU3m3xdmVU0Ryh0h25hTZyLUxJiBYuDbGHKaqvJ2WQbQrlAsHdz7u6Rvffp9L5oFi7r6gb619HaLCmDwymStTu/HZxn38e0k6D85dz+OfbeEnQ7pwUlyEezpHGIkxLiJCg3l04RY+WruXfp1jeO7a4QztZgsrmSNUlV25RZzdJ4GdOUUU2ZxrY0wAsHBtjAGcdnX3vf8d76101msa2zeRP186mK7tIhr9GnNWZhLtCmHcgM71HhMUJIwb2JlxAzuTtiOXfy9J590VmXUGo7DgIO46vw83n9PL+hebWrIPlVFcXknfzrHMX7+PIlul0RgTACxcG2PYX1DCLa+uYOWuPH59Xm9iw0P5+/zNjHv0f9wzoR/XnNb9mDfzFZZWMO+7vVw8pMtRPaIbktqjA6k9Ohw+P/tQKdmHSskqKCWnsIxRPePplRh9wj+faZ2qb2bs7174p6jcRq6NMf5n4dqYNm79nnymvJJGblEZz1w9jAsHnwTA+QM6cd/73/HAh+uZu3oP0y8/hZM71h90P1n3A0Vllfx0eFKT6ohyhRDlCqF7fNSxDzaGIz2ue3eKJkigyPpcG2MCgH3Oakwb9um6vfz02a9R4N1bzjgcrMFpKzfrhpH842dD2Lr/EBf+6wsWbd5f72u9uyKDHvGRDO/e3geVG3Nk5DqpfSSRYSE259oYExAsXHtZXl4ezzzzTJPOffzxxykqKjr2gcY0watf7+CW11bS76QYPrx9NIO61l55UET46fAkPrvzHHp1jObXb64+akW8ahm5RXyTnstPhyfZAirGZ3blFtE5Npzw0GAiw4Ipsm4hxrQZgZyvLFx7WSD/45u266O1e3hg7nrO69+J2VNG0TEmvMHjE2NcPHfNMKpUufX1FZTUmNs6Z2UmInDpsKZNCTGmKXblOm34AHe4tpFrY9qKQM5XNufay6ZNm8b27dsZOnQo559/Ph07duTtt9+mtLSUSy+9lIceeojCwkKuuOIKMjMzqays5A9/+AP79u1jz549jB07loSEBBYtWuTvH8W0El9uy+Y3b61mRPcOPHXVqYSHNu7mw+7xUTx2xVBumpXGH+euZ/rlpwBHeluf0Sv+uDqLGHOiMnKLDi/u40wLsZFrY9qKQM5XbSdcfzINfviueV+z82CYML3BQ6ZPn866detYvXo1CxYs4N1332XZsmWoKhdffDFLliwhKyuLLl268PHHHwOQn59PXFwcjz76KIsWLSIhwbsrw5m2Y93ufG5+dQU9E6KZcV1qo4N1tfMGdOK2sb14etF2hiW354oR3Vi+I5eM3GLuPL+Pl6o2praS8kp+OFhiI9fGBAI/ZKxAzlc2LcSHFixYwIIFCzj11FMZNmwYmzZtYuvWrQwePJiFCxdyzz338MUXXxAXV3vuq2m7fsgvoayi6oRfZ2dOIT9/aTlxEaG8csPIJi8Zfuf5fRl9cjx/+HAd63bn8+4Kp7f1BQPr721tTHPbnVeMKnSPd4drVwiFFq6NaZMCLV+1nZHrY4ww+4Kqcu+993LzzTfX2rdy5UrmzZvH/fffz7nnnssDDzzghwpNoNmVU8R5j/6PLu3C+d34fkwYdPyrJgJkFZRy3cxlVFRV8eYNo+gc1/Ac64YEBwlPTDqVi55cyq2vryD3UBkXndLFZ0uSGwPOfxvgdLUBiAwN5of8Yn+WZEzb5eeMFWj5ykauvSwmJoaCggIALrjgAmbOnMmhQ4cA2L17N/v372fPnj1ERkZyzTXXcPfdd7Ny5cpa55q26cWl6ShKWEgQv3x9JZc/+xUrdh44rtfIPFDEz19axr6DJcz8+YgGe1U3Vny0i6evHsYP+SUUllVyeRN7W5vAISLdRGSRiGwQkfUi8qs6jhEReUJEtonIWhEZ5rHvehHZ6v663tv1VrfhOzwtxBVMofW5NqbNCOR85dWhJhEZD/wLCAZeUNXpNfYnA68A7dzHTFPVeSLSA9gIbHYf+o2q3uLNWr0lPj6e0aNHM2jQICZMmMBVV13F6aefDkB0dDSvvfYa27Zt4+677yYoKIjQ0FCeffZZAKZOncr48ePp0qWL3dDYBuUVlfF2WiYTh3Zl+mWDeXdFJv9cuIXLn/2KCwd35ncX9KNHQv0LrlRWKa98tYN/LNiMAM9eM5xhyc3Xg3pYcnv+/tMhfLE1mxE9rLd1K1AB3KWqK0UkBlghIgtVdYPHMROA3u6v04BngdNEpAPwIJAKqPvcuap6fH8JHodduUVEhAaTEB0GOHOui22FRmPajEDOV6Kqzf6iACISDGwBzgcygeXAZM8LtYg8D6xS1WdFZAAwT1V7uMP1R6o6qLHvl5qaqmlpaUdt27hxI/379z/hn6WlaGs/b2v39KJt/H3+Zub/+mz6upd3LiytYMYX6Ty/JJ2S8krO7J3I5cO6Mm5A56OWHN+yr4B75qxl1a48xvRN5C+XDrZOHgFORFaoaqq/66gmIh8CT6nqQo9t/wYWq+ps9/PNwJjqL1W9ua7j6lPXdbuxpsxKY1dOEfN/czYAj8zbyMtf7WDznyc06fWMMcenLWWOun7Whq7Z3hy5HglsU9V0dxFvAhMBz1EQBWLdj+OAPV6sx5gWo7Sikpe+3MHZfRIPB2twlgj/9Xl9uGpkMq9+s5P3Vu7mV2+uJtoVwoRBnbl0WFe+Tc/lmcXbiHaF8NiVQ7hkaFdb2MUcF/cAx6nAtzV2dQUyPJ5nurfVt72u154KTAVITk5uco0ZuUWH51sDRIQFU1pRRWWVEhxk/383xviPN8N1XRfb02oc80dggYjcAUQB53nsSxGRVcBB4H5V/aLmGzTXRdqYQPPhqj1kHypl6lk969zfMTacu8b15Tfn9eHb73N5b2Um877byzsrMgG4eEgXHvzJAOKjXb4s27QCIhINzAF+raoHm/v1VfV54HlwRq6b+Brs8uhxDRDlvqG2qKyCmPCmdcIxxpjm4O/b+ycDL6vqP0XkdOBVERkE7AWSVTVHRIYDH4jIwJoX+sZcpFW1TYzaeWt6j/E9VWXGF+n0PymW0SfHN3hsUJBweq94Tu8Vz8MTB/HfTfvoEBV2VOgwprFEJBQnWL+uqu/VcchuoJvH8yT3tt04U0M8ty/2TpWQU1hGUVklyR2OTHWqnhZVVFZp4doYH2kLGasp+cqb3ULquwh7uhF4G0BVvwbCgQRVLVXVHPf2FcB24LhXqAgPDycnJ6fVB09VJScnh/DwprdXM4Fj8ZYstu4/xNSzU47rohURFsxFp3SxYG2aRJz/s70IbFTVR+s5bC5wnbtryCggX1X3AvOBcSLSXkTaA+Pc27zicKeQ+CPTQqJcR8K1Mcb72kLGamq+8ubI9XKgt4ik4ITqScBVNY7ZBZwLvCwi/XHCdZaIJAK5qlopIj1x7kxPP94CkpKSyMzMJCsr60R+jhYhPDycpCRrh9YazFiSTufYcC46pYu/SzFty2jgWuA7EVnt3nYfkAygqs8B84ALgW1AEfAL975cEfkTznUf4GFVzfVWoRk12vABRIQ6v84KS20JdGN8oa1krKbkK6+Fa1WtEJHbcUYvgoGZqrpeRB4G0lR1LnAXMENEfoNzc+PPVVVF5GzgYREpB6qAW5pyoQ4NDSUlJaXZfiZjmkNJeSUrdh5gZEoHQoOP/vBo3e58vtqew70T+tXaZ4w3qepSoMGPStQZorqtnn0zgZleKK2W6gVkktrXHrm2dnzG+IZlrPp5dc61qs7DGenw3PaAx+MNOKMlNc+bgzPvz5hWZd/BEqbOSmNNZj5d4sK54cwUJo1MJtrl/Kc444t0ol0hTD7NbtA1pj47c4voHBtOeOiR9pOR7jnXNnJtjPE3GxozxkfWZuZx8VNL2br/EPdO6EdyfCR//ngjZzzyX/726SbWZOTx0dq9TBrRjVi7IcuYeu3KLTpqSghApLtbSLHNuTbG+Jm/u4UY0yb8Z80efvvOGhKiXcy59Qz6nxTLzef0YnVGHs8v2c6z/9vOM4u3Exwk/OJM+5jNmIZk1GjDBx4j1xaujTF+ZuHaGC+qqlIe/+9WnvjvVkb0aM+z1wwnwaP39NBu7Xjm6uHsyC7kpS+/56R2EbaSojENKCmv5IeDJQ2MXNu0EGOMf1m4NqYZlFdWsT3rENkFZWQdKiGroJTsQ2V8l5nP1+k5/Gx4En++dBCukOA6z++REMVDEwf5uGpjWp7decWoQnL80X+E2si1MSZQWLg25gRlHijiplfS2PRDwVHbXSFBdIx18cBFA/jF6B6tvtG+Mb6wq442fAARodbn2hgTGCxcG3MC0nbkcvOrKyirrOKRywbTMyGKxBgXiTEuol0hFqiNaWbVPa671QjXQUFCRGgwRdYtxBjjZxaujWmid9IyuO/97+jaLoIXrh/ByR2j/V2SMa3erpwiwkODSPS4d6FalCuYIutzbYzxMwvXxhynyipl+icbmfHF94w+OZ6nrxpGu8gwf5dlTJtQ3Yavrk+FIsNCbOTaGON3Fq6NaaTKKmVNZh5P/ncrizZncf3p3bn/ogG2kqIxPlRXj+tqkWHBNufaGON3Fq6NacCevGKWbMliydYslm7N5mBJBaHBwp8vGcQ1o7r7uzxj2hRVrbPHdTUL18aYQGDh2pg6/G9LFv+Yv5nvducD0Dk2nPGDOnNW70TOPDmB9lE2DcQYX8spLKOwrJLkDnX3go8MC6HI+lwbY/zMwrUxHjbsOcgjn2zki63ZJHeI5PcX9uecvon07hhtnT+M8bPDbfji658Wkn2o1JclGWNMLRaujQH25hfzzwVbmLMyk9jwUP5w0QCuGZVc76Ivxhjf616VyaKerxPr6gp0qrXfpoUYYwKBhWvT5s36egd/nbeRqiqYclZPbhtzMnGRof4uyxhTQ3xIGfF7Poby6+vcH+myaSHGGP+zcG3aLFXl0YVbePLzbYztm8jDEwfVWpjCGBNAXDHO99KCOndHhtrItTHG/yxcmzapskr5w4freOPbXUwa0Y2/XDqY4CCbU23aLhGZCVwE7FfVQXXsvxu42v00BOgPJKpqrojsAAqASqBCVVO9UmR4rPO99GCdu52R60qqqpQg++/ZGOMnXm3QKyLjRWSziGwTkWl17E8WkUUiskpE1orIhR777nWft1lELvBmnaZtKa2o5P/NXsUb3+7i1jG9eOQyC9bGAC8D4+vbqap/V9WhqjoUuBf4n6rmehwy1r3fO8Eajj1yHebcI1FSYaPXxhj/8drItYgEA08D5wOZwHIRmauqGzwOux94W1WfFZEBwDygh/vxJGAg0AX4TET6qKpdMc0JKSyt4JbXVvDF1mx+f2F/ppzd098lGRMQVHWJiPRo5OGTgdneq6YeoZEgQVBS98h1lDtcF5ZWEhlmH8waY/zDmyPXI4FtqpquqmXAm8DEGsco4P6cjzhgj/vxROBNVS1V1e+Bbe7XM6bJDpVWcNUL3/LV9hz+/tNTLFgb0wQiEokzwj3HY7MCC0RkhYhM9eKbO6PX9YxcR7gDdbHNuzbG+JE3/7TvCmR4PM8ETqtxzB9xLsh3AFHAeR7nflPj3K4138B9EZ8KkJyc3CxFm9brucXbWZORx3PXDGf8oM7+LseYluonwJc1poScqaq7RaQjsFBENqnqkrpOPuHrtiuu3nB9eOTaOoYYY/zIq3OuG2Ey8LKqJgEXAq+KSKNrUtXnVTVVVVMTExO9VqRp+fYdLOGFpen8ZEgXC9bGnJhJ1JgSoqq73d/3A+/TwCeNJ3zddsXUe0NjhDtcW8cQY4w/eTNc7wa6eTxPcm/zdCPwNoCqfg2EAwmNPNeYRnv8sy1UVil3j+vr71KMabFEJA44B/jQY1uUiMRUPwbGAeu8VkQD4TrK5XwYa72ujTH+5M1wvRzoLSIpIhKGM9oxt8Yxu4BzAUSkP064znIfN0lEXCKSAvQGlnmxVtOKbdtfwFvLM7j6tO71LptsTFsnIrOBr4G+IpIpIjeKyC0icovHYZcCC1S10GNbJ2CpiKzBuU5/rKqfeq3Q8Nj651yH2si1Mcb/vDbnWlUrROR2YD4QDMxU1fUi8jCQpqpzgbuAGSLyG5wbYn6uqgqsF5G3gQ1ABXCbdQoxTfV/n24mMiyEO350sr9LMSZgqerkRhzzMk7LPs9t6cAQ71RVB1cM5Gyvc5eNXBtjAoFXexWp6jyc9nqe2x7weLwBGF3PuX8B/uLN+kzrl7Yjl4Ub9vHbcX2Ij3b5uxxjzIlqoFtIpEcrPmOM8Rd/39BojNeoKn+dt5GOMS5uODPF3+UYY5qDq/5pIdXh2lrxGWP8ycK1abXmr9/Hyl15/Ob8PraghDGthSsWKoqhsrzWrur/zq0VnzHGnyxcm1aporKKv83fRK/EKH42PMnf5RhjmksDS6AHBwmukCAbuTbG+JWFa9PqFJZW8PJXO0jPKuR34/sREmz/Nzem1TgcrutuxxcZFmwj18YYv7LPyk2L9um6H/hw9W72F5SSfaiUrILSw224Uru3Z9yATn6u0BjTrMJjne/1zrsOsVZ8xhi/snBtWqR9B0t44MN1zF+/jy5x4fRIiGJIUjsSY1wkRLtIjHFxfv9OiIi/SzXGNKfqkeuS+keui6xbiDHGjyxcmxZFVXlreQZ/mbeRsooqpk3ox01nptjUD2PaigbmXANEukIoKrdwbYzxHwvXpsXYkV3Ive99x9fpOYzq2YFHLjuFlIQof5dljPElV5zzvZ5wHRUWTFGpzbk2xviPhWvTIizZkt80PTUAACAASURBVMXUV9MIDQrikcsGc2VqN4KCbMqHMW3O4ZHr/Dp3R4YFk1dUu02fMcb4ioVrE/CWbs1myqw0eiZG89LPR9A5LtzfJRlj/OVY00LCQmz5c2OMX1m4NgHtq23Z3DRrOSkJUbx+02l0iArzd0nGGH8KjQAJbnCVRusWYozxJ7sLzASsb9JzuPGVNJI7RFqwNsY4RJx2fNaKzxgToCxcm4C07Ptcbnh5OV3bR/D6TaOIj3b5uyRjTKBwxTTciq+sAlX1cVHGGOOwcG0CzoqdufzipWV0jgvnjSmnkRhjwdoY48HVwMi1K5gqhdKKKh8XZYwxDgvXJqBk5BZx4ytpdIwNZ/aUUXSMsZsXjTE1uGLrX/48NBjApoYYY/zGq+FaRMaLyGYR2SYi0+rY/5iIrHZ/bRGRPI99lR775nqzThMYSsor+eXrK6msUl76+Qg6xVqwNsaXRGSmiOwXkXX17B8jIvke1+YHPPY1eL1vVq6Y+sO1y7lPv9B6XRtj/MRr3UJEJBh4GjgfyASWi8hcVd1QfYyq/sbj+DuAUz1eolhVh3qrPhN4HvrPBr7bnc+M61LpYYvDGOMPLwNPAbMaOOYLVb3Ic0NjrvfNyhUDOVvr3BUZ5oxcF9sqjcYYP/HmyPVIYJuqpqtqGfAmMLGB4ycDs71Yj/GByiqlvPL45zq+k5bB7GW7+OWYXpw/oJMXKjPGHIuqLgFym3Dq8V7vT4wrpoEVGm3k2hjjX94M112BDI/nme5ttYhIdyAF+Nxjc7iIpInINyJyST3nTXUfk5aVldVcdZsmKigp56Inl3Ldi8uO60799Xvyuf+DdZzRK547z+/jxQqNMc3gdBFZIyKfiMhA97ZGX++bRXhsvd1CIqpHrm3OtTHGTwLlhsZJwLuq6nk17K6qqcBVwOMi0qvmSar6vKqmqmpqYmKir2o1daisUu6YvYqNew/ydXoO/1m7t1Hn5ReVc+trK2kfGcYTk08lJDhQ/i9pjKnDSpxr8xDgSeCD432BZhkUccVAZSlUlNbadXjk2sK1McZPvJlkdgPdPJ4nubfVZRI1poSo6m7393RgMUfPxzYB5q/zNrJ4cxZ/mjiQgV1imT5v4zFHjqqqlDvfXs3e/GKevnoYCdbL2piApqoHVfWQ+/E8IFREEjiO632zDIq4Yp3vpYdq7aoeubYl0I0x/uLNcL0c6C0iKSIShhOga3X9EJF+QHvga49t7UXE5X6cAIwGvHNjjDlhby7bxYtLv+fnZ/Tg2tN78MBFA9iTX8KML9IbPO/Jz7fx3037uf/HAxjevb2PqjWmmVRVQe73sPlTWPoYvH8L/PsceO9mf1fmNSLSWUTE/Xgkzu+QHBp5vW82h8N17akhUS5rxWeM8S+vdQtR1QoRuR2YDwQDM1V1vYg8DKSpavWFdxLwph49Sbc/8G8RqcK5eE/32l3n5oR8vT2H+z9Yx9l9Ern/x/0BOK1nPBcO7syzi7dzRWo3OsfVbqn38dq9PPbZFi4b1pXrTu/u67KNaZqSfCdMb/gA0hdDedGRfTFdILEvdOzvt/JOlIjMBsYACSKSCTwIhAKo6nPAT4FbRaQCKAYmua/ddV7vvVaoK8b5Xke4jgx1fq1ZuDbG+IvXwjUc/thwXo1tD9R4/sc6zvsKGOzN2syJ25lTyK2vr6B7fCRPXXX0fOl7J/Tnsw37+dv8TTx6xdEdFb/LzOeud1YzvHt7HrlsMO6BMGMap6wItI7gFBIBwV64pBXnweZPnEC9/XOoLIPYrjD0aug82AnTCX0gol3zv7ePqerkY+x/CqdVX137al3vveZwuK7dMeTwtBDrFmKM8ROvhmvTeh0sKefGV9IAePH6EcSGhx61v1uHSG48K4VnF2/nutN7MLSbEzz2HSzhplnLiY9y8e9rh+MKCfZ57aaFKdgHO5fCDvdX9pa6jwuNhG6nQY8zocdZ0OVUCAk7+hhVKCt0RqDR2vsKsyBrk/O13/09b5dzbGwSjJgCAy+BrqkQZDff+k149bSQ2uE6LCSI0GChyPpcG2P8xMK1OW4VlVXc/sYqdmQX8uqNp9W74MttY0/mnbRMHv7PeubcegYl5VVMmZVGQUkFc249w25gNI7yYti/AQqzna+ibCfkHsqC3SuOLBYSFgPdT4fBP4PQiNqvk7cLdnwJn//JeR4SAd1GQHCY+3VznO8VxceuKSgUEnpD1+HOCHWvsRaoA0n1nOt62vFFhoXYyLUxxm8sXJvj9pd5G1myJYvplw3m9F7x9R4X7Qrhdxf05Xdz1jJ3zR4WbNjHd7vzef7aVPqfFOvDik1AytkOaTNh1WtQknf0vmAXRCVAp4Ew7FpnNLrzkMZN+yjMgZ1fOqPcGd8A4rxWx/4QGe88jmgPUkdQDo+DxP7Qoad3ppiY5tHAnGtwVmm0OdfGGH+x3x7muLz+7U5e+nIHN56ZwqSRycc8/vLhSbzy9Q7ufmctZZVV3Duhn63A2JZVVcKW+bD8Bdj+XwgKgf4Xw6DLIOakI+E3LBqaOhc/Kh4GXOx8mdapgTnXYOHaGONfFq5No321LZsHP1zPmL6J3Hdh4zoiBAcJD1w0gEkzvuHyYUlMPbunl6s0XqXqjBZ6TuEICYeUcxoe6S0vgVWvwpdPQP4up7PG2N/DsOshxv7YMscpJNyZulPvyHUIhdbn2hjjJxauTaN8n13Ira+vJCUhiicmn0pwUONHFU/rGc+Su8fSpV2EdQbxp/JijznNOc685opiiOjgjBZHJkBUojNlojQfsjYffWNf9lYo3O90y6gprhsM/zkMuw6iO3q8ZwmsnOX0gS7YA91Gwfi/Qp8JNu3CNJ2IM3ptI9fGmABkv93MMeUXlXPjy8sJkro7gzRGtw6RXqgswFWUwvr3neCacg4EH///bsdNFfIz3cF445FwnL3VCcyNIUGgVUeeh0Y6/Zt7nAkxnT2CeIIzjSM/05nm8fmfYPF0GDARUm+Afeth6aNQsBeSz4BLn4OUs5s+3cMYT8cI11mHai+NbowxvmDh2jSouKyS295YScaBIl678TSS49tgSD5eqrDlU5h/H+S6V6mMaA/9fgwDLoWezRC0K8ud0Jq1GfZv9AjTm6HMY0noqI5OMD7lZxDbxSMUu7+HRkBRrns022OqR1i0cwNgYj9nVLqhLhldhznzm7O2ODcorn4D1r3r7Es+Ay573mmNZ6HaNKfw2PrDtSuEotyiOvcZY4y3Wbg29fohv4Qps9JYtyefv11+Cqf1rL8ziHHL2gKfTnNu1kvoC1e9A1UVzgIkG+Y6nTHC20Gf8XDSKU54TeznBN/q8KkKh/YfCctZm+HQPnf4zXLCb0mNUeiojtCxHwy9ynm96mAc2eHYNcd2aZ6fPbEPTJgO5/4BNn3svG730RaqjXe4YuttxRcVFkxRqU0LMcb4h4VrU6fVGXlMnZVGYWkFM65N5Tzr8NGw4jz4399g2b8hNAoueARGTjkyQt3vQmeayPbPYf0HsO0zWPvmkfNdsc4Ic1CIMxLt2ZrOFecE1agEJ5BXjzpHd3LOaWyI9pWwKDjlCn9XYVo7Vwwc3FPnrsiwEIrshkZjjJ9YuDa1fLh6N3e/u5aOMS5evXE0fTvH+LukwFWcB98+B18/43QuGHYdnPuAE35rCnFB3wnOFzgj0VmbPKZ1bHJa1Q281D367B7Vju5ko7/G1GQ3NBpjApSFa3NYVZXy6MItPLVoGyNTOvDs1cOIt1UU61acB98863yV5kO/i2DMNOg8uPGvEZUAUWc6NwoaY46PK7bBRWQqqpSyiirCQmxVTWOMb1m4Nof9/oN1zF62i0kjuvHwxEH2S6kuxQc8QvVBJ1Sfc48zXcMY4zsNjlw7v9qKyioICwnzZVXGGEOj0pOIXCoicR7P24nIJd4ry/jaki1ZzF62iylnpfDIZYMtWNdUfAA+/ws8fgr87/+cjh+3LIVJr1uwNgFHRH4lIrHieFFEVorIOH/X1axcMU7P9YraLfciw4IBbGqIMcYvGpugHlTVw+0JVDUPeNA7JRlfKyqr4L73v6NnQhR3jevb9hZ6UXVa29WlKBc+/7MTqpf8DXqOcUL1la8d3xQQY3zrBlU9CIwD2gPXAtP9W1IzC3eP99TRMSTSdWTk2hhjfK2x00LqCuE2paSVeGzhFjIPFPPW1FGEhwb7u5wTV1XpzIkOi4LQ8LqPUYU9K53OHRs+hLydzhxOzx7QYdGw+RMoK3AWRjn7d9B5kG9/FmOapvov5AuBV1V1vbS2v5pd7hutSw9CdOJRuyJDbeTaGOM/jQ3IaSLyKPC0+/ltwIpjnSQi44F/AcHAC6o6vcb+x4Cx7qeRQEdVbefedz1wv3vfn1X1lUbWao7Dd5n5vLj0eyaPTG55faxVYf17Tv9ozx7QRbmAOisNtk9x93zuC4n9nV/C2z5zB+pdTuu7nmNgyGSn/V1hlvNaebugKAdOPhfO+R10GujnH9aY47JCRBYAKcC9IhIDVDV0gojMBC4C9qtqrb8iReRq4B6c4F4A3Kqqa9z7dri3VQIVqprajD9L3Q6H69rzriNdTrgutF7Xxhg/aGy4vgP4A/AWoMBCnIBdLxEJxgnj5wOZwHIRmauqG6qPUdXfeBx/B3Cq+3EHnGknqe73W+E+90Aj6zWNUF5ZxT1z1pIQ7WLahH7+Luf47F0Dn0yDXV9BbBK0S3YWMYka7Yw8R8Y74bh6IZbNn4C6f9EGhUDPsc6NiH0vDKwe0cY0jxuBoUC6qha5r6m/OMY5LwNPAbPq2f89cI6qHhCRCcDzwGke+8eqavaJlX0cGgrX7hsai8ttWogxxvcaFa5VtRCYdpyvPRLYpqrpACLyJjAR2FDP8ZM5Mo/7AmChqua6z10IjAdmH2cNpgEvLv2eDXsP8tw1w4iLOMHluH2lMBs+/xOseMUJxT/5F5x6LQQdYzpLRSnkbIeDuyEp1VmO3JjW63RgtaoWisg1wDCcTxHrpapLRKRHA/u/8nj6DZDUDHU2nSvW+V5HO76oMBu5Nsb4T2O7hSwUkXYez9uLyPxjnNYVyPB4nuneVtfrd8f5+PLz4zlXRKaKSJqIpGVlZR37BzGH7cgu5LGFWxg3oBPjB53k73KOraLUWajliWHOEuKjboU7VsLwnx87WIOzgEunAdD7fAvWpi14FigSkSHAXcB26h+RboobgU88niuwQERWiMjUZnyf+jUwch3hDtfFNufaGOMHjZ0WkuDuEAKA+2PBjs1YxyTgXVU9riuhqj6P89Ekqamp2oz1tGqqyn3vf0dYcBAPTwzwG/QqSmHlLFj6mDPq3OtHMH66M4faGFOfClVVEZkIPKWqL4rIjc3xwiIyFidce65+dKaq7nb/XlgoIptUdUk9508FpgIkJyc3vZDDI9e1w3WUe1pIoXULMcb4QWNb8VWJyOGroPujw2OF2d1AN4/nSe5tdZnE0VM+judcc5xe+nIHX23P4Z4J/egcV083DX8rL4FlM+BfQ2HebyGuG1z7PlzzngVrY46tQETuxWnB97GIBAEnPPdLRE4BXgAmqmpO9XZV3e3+vh94H2daYJ1U9XlVTVXV1MTExPoOO7Zwd7guya+1K8L6XBtj/KixI9e/B5aKyP9w7hQ/C/fIQwOWA71FJAUnGE8Crqp5kIj0w+nD+rXH5vnAX0Wk+vP7ccC9jazVNGDx5v38+eMNnD+gE1eNPIFRI2+prICVr8CSf0DBHkg+HS59FlLOgVbWScwYL7oS53p7g6r+4B4c+fuJvKD7Nd4DrlXVLR7bo4AgVS1wPx4HPHwi79UoIS4IDqtz5NoVEkRwkFifa2OMXzT2hsZPRSQVJ1CvAj4Aio9xToWI3I4TlIOBme5eqw8Daao6133oJOBNVVWPc3NF5E84AR3g4eqbG03TbdtfwB1vrKJv51gev3IoQUEBFla/X+J0ANm/3h2qn4OUsy1UG3Oc3IH6dWCEiFwELFPVBudci8hsYAyQICKZODeYh7pf7zngASAeeMbdMru65V4n4H33thDgDVX91Cs/WE31LIEuIkSGBtvItTHGLxoVrkXkJuBXONMzVgOjcEaaf9TQeao6D5hXY9sDNZ7/sZ5zZwIzG1OfObYDhWXc8HIartBgXrg+lShXAK0BdGAnLPg9bPyP01Lvileh/08sVBvTRCJyBc5I9WKcTxufFJG7VfXd+s5R1ckNvaaq3gTcVMf2dGDICRXcVK6YOruFgNPrusi6hRhj/KCxCetXwAjgG1Ud657K8VfvlWWaU1lFFbe8toIfDpYwe8oouraL8HdJjvJi+OKf8OUTTsePsffDGbdDaIDUZ0zL9XtghHsONCKSCHwG1BuuWyRXbJ0j1+D0ui4qt3BtjPG9xobrElUtERFExKWqm0TE7iprAVSVB+eu49vvc3n8yqEM7x4gbegqK+Cta2HbQhj8MzjvIYirs1OjMeb4BVUHa7ccGn8De8vRYLgOpqjU5lwbY3yvseE6093n+gOcNksHgJ3eK8uciPLKKnIOlZF9qJQFG/Yxe1kGt489mUtODZDwqgqf/M4J1hc9Bqk3+LsiY1qbT91rEVR3YbqSGlP0WgVXDBzMrHNXZJjNuTbG+Edjb2i81P3wjyKyCIgDfHPDijlKRWUVy77PZW9+CVmHSskuKCXrUClZBc5X9qFSDhSVH3XO+IGdufP8Pn6quA7fPANpL8IZ/8+CtTFeoKp3i8jlwGj3pudV9X1/1uQV4bGwv54512Eh5BWV+bggY4xp/Mj1Yar6P28UYhqmqizavJ9H5m1i6/5Dh7dHhAaTGOMiITqMlIQoRqZ0IDHG5d7momOMi1OS2gVOZ5CNH8H830P/i52pIMYYr1DVOcAcf9fhVfV0CwFn5HpPno1cG2N8L4BaRpj6rNudz18+3sjX6TmkJETx5ORTGdw1jsQYV2B1/TiW3Sthzk3QdThc9jwEtb4poMb4k4gUUPcCXwKoqsb6uCTvqg7XqrW6C0WGhdi0EGOMX7SgZNb27M4r5h/zN/P+qt10iArjoYsHctVpyYQGt8BQmrcLZk+C6ESYPNs6ghjjBaoa4+8afMoVC1XlUFFS65rizLm2GxqNMb5n4TpAlZRXcunTX5JfXM6tY3px65hexIaf8OrF/rFvPbx7g7Ok+XVzIbqjvysyxrQGLvffEqUFtcO1K5hCG7k2xviBhesAtWDDPvYXlPLyL0Ywpm8LDaNFubDor87Ni+FxMOk16NjP31UZY1oLl3uWS2lBrT/ao8JCKKuooqKyipCW+GmfMabFsnAdoN5enkHXdhGc3TvR36Ucv8oKWPESLPoLlORD6o0w9j6I7ODvyowxrUn1yHVJfq1dkWHBABSVVxJr4doY40MWrgNQRm4RS7dl8+vzegdOl4/G+n4JfDIN9q+HHmfBhP+DTgP9XZUxpjUK9xi5riEyzPn1VlRa2XKn1BljWiQL1wHonbQMROBnqd38XUrjHdgJC+6HjXMhLhmumOW025MW9seBMabl8JxzXcPhkWu7qdEY42MWrgNMZZXyzopMzuqdSNd2LaCjRlkhLH0cvnoCJAjG3g9n3G7dQIwx3teocG03NRpjfMvCdYD5YmsWe/NLuP/HA/xdSsNUYd0cWPgAHNwNg3/mLAoTFyBLrBtjWj9XnPO9tPYqjYenhVi4Nsb4mIXrAPN2WgbtI0M5b0AAdwipqoKPfwMrXoaThsBPZ0LyKH9XZYxpa1zRzve6wrXLGbkutGkhxhgfs3AdQHIOlbJwwz6uO70HrpBgf5dTt6oq+OhXsHIWnHkn/Oh+CArQWo0xrVuIC4JdDU4LKbaRa2OMj3m1P5GIjBeRzSKyTUSm1XPMFSKyQUTWi8gbHtsrRWS1+2uuN+sMFO+v2k15pXLliAC9kbGqCv7z/5xgffbv4NwHLFgb04qIyEwR2S8i6+rZLyLyhPuavlZEhnnsu15Etrq/rvdZ0a4YKKk9ch3lnhZSWGoj18YY3/LayLWIBANPA+cDmcByEZmrqhs8jukN3AuMVtUDIuI5F6JYVYd6q75Ao6q8tTyDod3a0adTAK5gXFUF/7kDVr0G59wDY+61TiDGtD4vA08Bs+rZPwHo7f46DXgWOE1EOgAPAqmAAivc1/sDXq84PLbOkeuI6pHrchu5Nsb4ljdHrkcC21Q1XVXLgDeBiTWOmQI8XX0BVtX9XqwnoK3clcfW/YcCc9S6qgrmVgfrac6CMBasjWl1VHUJkNvAIROBWer4BmgnIicBFwALVTXXfT1fCIz3fsU4I9d1hOsjI9cWro0xvuXNcN0VyPB4nune5qkP0EdEvhSRb0TE82IcLiJp7u2X1PUGIjLVfUxaVlZW81bvY28vzyAyLJifDOni71KOVl4CH/4SVlcH63v9XZExxn/qu6435nrvHa7YOm9oDA8NQgSK7YZGY4yP+fuGxhCcjxfHAEnAEhEZrKp5QHdV3S0iPYHPReQ7Vd3uebKqPg88D5Camqq+Lb35FJZW8NHaPfx48ElEu/z9T+IhY7kTrLO3wJj7YMw9/q7IGNPCichUYCpAcnLyib+gKxbydtb1PkSGBnOwxMK1Mca3vDlyvRvwnOOQ5N7mKROYq6rlqvo9sAUnbKOqu93f04HFwKlerNWvPl67l8KyysCZElJeDAv+ADPHQVkRXPOeBWtjDNR/XW/M9R5wBkVUNVVVUxMTE0+8IldMnSPXACmJUWzdX3vKiDHGeJM3w/VyoLeIpIhIGDAJqNn14wOcUWtEJAFnmki6iLQXEZfH9tHABlqpRZv307VdBMO7t/d3KZCxDJ47y1lxcdh18Muv4eRz/V2VMSYwzAWuc3cNGQXkq+peYD4wzn3tbg+Mc2/zvnrmXAMMSWrH2ox8qqpa7AebxpgWyGtzEFS1QkRux7nABgMzVXW9iDwMpKnqXI5ckDcAlcDdqpojImcA/xaRKpw/AKZ7dhlpbVZn5DGiRwfE3zcJLn0MPnsI4pLg2veh14/8W48xxqdEZDbOgEeCiGTidAAJBVDV54B5wIXANqAI+IV7X66I/AlnUAXgYVVt6MbI5lPdik+11o3WQ7q14/Vvd5GeXcjJHaN9Uo4xxnh1gq+qzsO5GHtue8DjsQJ3ur88j/kKGOzN2gLFvoMl7M0vYUi3dv4tZP378NkfYeClcPGTzi8sY0yboqqTj7Ffgdvq2TcTmOmNuhoUHgta6UxnC4s8atdQ93V1bWaehWtjjM94dREZc2yrM/LoRC6jXduPfbC37FsPH/wSkkbCpf+2YG2MaTmqr1d1TA3plRhNVFgwazLyfFyUMaYts3DtZ2sy8rgn9C36fjoZDu7xfQFFufDmVc4d91e+6iwnbIwxLYUr1vlex02NwUHCoK5xrM7M93FRxpi2zMK1n63OyGNkaDpSWQZfPeXbN6+qhDk3OqH+ytcgprNv398YY05UA+EanKkhG/ccpLTCFpMxxviGhWs/qqpSvs/cQ1LVbgiJgLSZUJjtuwL++xBs/xwu/Ad0G+G79zXGmObSwLQQcG5qLKusYtNea8lnjPENC9d+tD3rECnlW50n5/0RKkrgm2d88+bfvQtf/gtSb4Th1/vmPY0xprlVh+uSukeuq28WX5Np866NMb5h4dqPVmfkMUTSnSenXAEDJsKyGVDsxV8ClRWwejZ8eDsknw7jp3vvvYwxxtvCq6eF1D0y3SUunIRoF2sybN61McY3LFz70eqMPIaFpKPtUyCyA5x1lzNvcNmM5n+zygpY/QY8PQI+uAUS+8AVsyAkrPnfyxhjfMXVcLgWEYZ2i7ORa2OMz1i49qM1mU64lq7DnA0nnQK9L3CmhpQeap43qayAVa/DU6nwwa0QFgVXvg5TFkN0x+Z5D2OM8Zcwd//qesI1wClJ7diedYiDJeU+KsoY05ZZuPaTkvJKsvdmEF+ZDV2GHdlx9m+hOBdWvHRib1BZDqtec0L1h7905iVOegNu/gL6XwRB9k9vjGkFQsIgJBxK65/2MaRbO1RhnbXkM8b4gCUsP1m/J5+BbHOedB1+ZEe3kZByNnz1JJSXHP8LV5bDylfdofo2Zz7ipNlw8xLo9+NaywMbY0yL54ptcOR6SFIcAKttaogxxgcsXNflzavh0/u8+harduVxSlA6KkHOdBBPZ/0WDu2DVa82/gUry2HlLHhyOMy9HcLbweQ3Yer/oN+FFqqNMa2XK6bBcN0uMowe8ZG2UqMxxicsXNdUVQXbF8GWT7z6Nmsy8xkZugNJ7O/Mg/aUcjYkjYAvn3BCc0MO7oXF0+HxwTD3DufGyMlvwdTF0HeChWpjTOvniqm3FV+1Id3aWccQY4xPhPi7gIBzMBPKCyE33WmJF9HOK2+zelcuj8h26Hpx7Z0icPbd8MYVzgqK3UdDYj/o2B+iEp1jdiyF5TNg40eglXDyefCTJ6D3+RaojTFtS3jD00LAuanxw9V72HewhE6x4T4qzBjTFlm4rilr85HHe9dAz3Oa/S1yDpWiebuIdh08+mZGT73HwdCrnfC84cMj2yPaO6M0ebucqR+jboXUGyC+V7PXaYwxLYIrFgq/b/CQod2ceddrMvIYN7CzL6oyxrRRFq5rOipcrz4crg8UlnH9S8u48/w+jOl7Yi3s1mR6LB7TtZ5wLQKXPAMTn4aCHyBr05Gvgn1wzjQYdBmERpxQLcYY0+K5YqCk4SkfA7vEERwkrMm0cG2M8S6vzrkWkfEisllEtonItHqOuUJENojIehF5w2P79SKy1f3lu/W5szY5Uy/ikmHP6sOb03YeYG1mPre+tpJVuw6c0FuszshnaNB2NDgMOg5s+GARiD0Jeo11Rql/8i+46k049WoL1saY/9/efYfHVVyNH/8ercqqd6vLvffegFCNCWAIvYWO00hCSeMXAoF0eBOSvC9J6C1gWiiGEMChgws2tmzjbslFkiWrWVYvq53fH7OS15JsC2tXZXU+avoMfwAAIABJREFUz3Ofu3v33tWMvbo6O3NmRoFNm6sqsLMsHYEzxMGY1GjNu1ZK+Z3fgmsRcQAPAmcB44DLRWRcu3NGAncA840x44FbPMcTgLuB2cAs4G4RifdXWQ9Tus3eqNMnw751bYe3FtnBMknRoVz/5GpyS49/kZec/Epmh+1GUifpColKKdVd874P486Hd++ENY8f8bTJWXGsL6jE3dLSg4VTSg00/my5ngXsNMbkGWOagOeB89qdcxPwoDHmAIAxpsRz/ExgmTGmwvPaMmChH8tqGQNl2yBpFKRNgQO77KBGYGtxNdkJEfzzhtk4goSrH/uc4oNffR5qYwwb95Yz2uQdOSVEKaV6wbF6G0XkARHJ8WzbRaTS67UWr9eW9mjBgxxwwSN2hds3b4P1L3R62qxBhl+4HkR+nQxPnA1fPNV2j1dKKV/xZ851BpDv9bwA2xLtbRSAiHwGOIBfGmPePsK1Ge1/gIgsBhYDZGdnd7/ENftt3l7ymEMDBD2DGrcUVTEmNZrBiZE8ed0sLn1oBdc8/jkvfnsuseEhXf4Ru8vrSGrcS1hY/ZEHMyqlVA/z6m08A3vPXS0iS40xm1vPMcbc6nX+94GpXm9Rb4yZ0lPl7SA4FC55Cp69GF77DoRGwNhz7WvGQM5znPPJzzGOKvIzzia7ZjO88QN468cw+iz2Dz2PspQTGZ+d3GtVUKpfqCmFrW+Au8UzO5mABNnH4oCgYM/mOLRvaQZXI7Q02r2rERyhNuU1aWRv18jnentAYzAwEjgZyAQ+FpGJXb3YGPMw8DDAjBkzTLdLU7rV7pNHQapnYZd966jPPIFd5bWcOzkdgAkZsTz0zRlc9+Tn3PTUGp6+YRbOEEeXfsT6/EomB+XaJ9pyrZTqO9p6GwFEpLW3cfMRzr8cm77Xd4SE28WznvkGvHSdHZ8Smw1v3gp7PsWROZvz9l7EtEHz+eW542DfWhq/WIJrw8ukbH6NepPCv2b/lQvOOhPRKU2VOlxLM3z+iF1bo9GHYxcSR9h1OUadBVmzwdGDoen2d2DTa3byiCDfJXP4swaFQJbX80zPMW8FwCpjTDOwS0S2Y4PtQmzA7X3th34raavS7XafPMYuxhKXDUU5bN9fjTEwNi267dQTRibxp0um8IPn13Hbizn87crpR3jTw+XkVzLNsQsTGo0kBt63NaVUv9WV3kYARGQwMBR43+uwU0TWAC7g98aY145wrW97HNsLi4IrX4KnzrGr7bpb7EJd5/4FmXo1zkdWkZNficttWLI3kT/lLKCm/kR+MaqARQV/5KxVV/NMwU+55LpbutxoolSfU11sx42Fx0Pa5O5PgJD7PvznZzZ1dvhpcMY9EJ0Gxm17hozbs7XY3zl3C7hdhzZHCAQ7ITgMHGF2X38AdrwL296Clf+wA5LD42H8N2Duzf6dYtjtho/+AB/9HlInQkOljft8xJ/B9WpgpIgMxQbLlwFXtDvnNWzrxxMikoRNE8kDcoHfeg1iXIAd+OhfpVvBGQtRKfZ52hTYl8PWYjuYcUxqzGGnnzs5nbzSWh7473Y27TvI+PTYY/6InPxKrgrbjaRP8em3JKWU6kGXAS8bY7xHBg42xhSKyDDgfRHZaIzJbX+hz3scOxMeB998zQbXCUPhjF9BlE33mJIVx5Of7ebrf/2E7ftrmDMsgV+cM5vx6bGY6gspeuRSrt53L6/8cSOzF/8vGQnRx/hhSvUyVxPsWwsFq6Fgjd2qCg69HhQMKRPsys+ZM2xs44w5FOQGh9lzRGxQ3Ja+0QS1pfDh72DrmxA/1PYMjVrom8XqnDEw6ya7NVTZAH7rv2HdP+GLJ2HceTD/Fkj3cbZZXQW8shh2LoPJV8A5f/L57Gt+C66NMS4RuRl4B5tP/bgxZpOI3AusMcYs9by2QEQ2Ay3Aj40x5QAi8itsgA5wrzGmwl9lbVO6DZJGH/rQpE+BLUvZlV9IRKiD7ISIDpdcPXcwD36wkxdX53PPeUcPrhtdLezYV86Q0F2Q7v/xmUop9RV0pbex1WXA97wPGGMKPfs8EfkQm4/dIbjuMZFJcMM7HQ5Py47j4Y/dNDS7+cdV0zlzfEpbCohEp5L+g2XkP38rF+z8J5//dQf7L32SaWO1l1H1McZA/uew4Xn48hXb8gq2xz17NmR8z6ae1lV4gu7VsH6JXdm5MxIEiG15bi8kEk67G+Z+zwbi/uCMgfHn223Br2Dl3+3MP5tehWGnwAm3wtCTuh/UF22AF66Cqn1w9h9hxg1+WdXar4ktxpi3gLfaHbvL67EBbvNs7a99HDjynEr+ULbNfiNrlW7H6jQXrGN06mSCgjr+B8RHhnLmhFRey9nHHV8fe9RuxHV7Kxnu3k2waYaMrqWRKKVUD+lKbyMiMgaIB1Z4HYsH6owxjZ5eyPnAfT1S6q9owbhUnrlhFrOGJhAW3Mn9OjiUrKseZP/H05ny/k8ofX4ha6ffxrRTL2lr/Vaq11Tk2dlwNrxgZzQLDocxZ9tUiqxZENXJIndjvm737hbbQ79/EzTX2ZZpV8Ohlmrj9rRmhx7aB4fD8FPtehs9JTrVpp2ceJsNsFf8DZ5eBAnDYdKlMOliSBh27PcxxtavoQoaq2DPZ/Cfn0J4Alz3H8ia6bcq9PaAxr6jttx2fySPOXQszXZFRFd8yZhJJx7x0ktnZPHG+n28s6mY86Z0mNSkzaOf7GKOcw8YdDCjUqpP6WJvI9ig+3lP40irscBDIuLGTvH6e+9ZRvqSoCDhxJHHDpJTTrqemuyJBD/zTaat/X+YtT9H0qfCyDNg5ALb+BJ0jJzs5nqoKYG6Mrs4WUymb9IBjbFB0s737D4sxqY0hsfZvTPWtmAmje7ZwWHKf4o2wMf3wZY3AIGhJ8JJP7Yz4jhjjnk5YD+vKePt1h84Y22L9ezvwJcv2y8UH/4OPvwtZM6CSZfY1uyqQqjYZb94tO5rS2xQ7W4+/D0HnwAXP9H5lxAf0t+6VmWeZc+TRx86FpGAKyabEQd2kph25Ly7ecMTyYwP54XV+UcMrrcVV/PfLfv5d3Yx1CRBbFan5ymlVG85Vm+j5/kvO7luOdDlmZ76i6ghMzE/Ws8P//4cwyuXc71rJ1Ef3WcHQoVGgTPO5mqGhENIhN23NNtpXWtKOs6o4AizLW6Jw+0WnQ5NNXYKWO/NEQpxWfbvRFyWnfEkMgkKv4DcD2xuavU++56RyTaIb+pkYbNgpw2k0qbYQW1pkyBxpB30qfqHfevgo/vsoL+wWBtQT78WYjN7u2Q9J8QJU6+y28EC2OgJtN/60eHnBTttXnjCUBg81/OlM+bQl8+IRBj6tR75wqnBdavSToJroCJ2HBMr11KSduRvhkFBwsXTs3jgv9vZW15HdmLH3OyHPsolItTBGPdO22qt0zwppVSfFx0exp03XcFF/xjCE2XN/Ou6MQw7uAoKPoemWtu93lxv9001nsFj421XetQgO0A+ItG2pJXn2q1sh50loaXJ/pDg8EMtzs4Y25Wdv9IG2u05Y2HYyXbGhuGn2uAboMVlu74bKu3COOW5UJRj12rY+BKseezQe0SleoL8YXafMNyzHxZYgXdjNRRvtGkQQcF2NojwBM8+3m7Bzr7x99gY+/mpK7d50tXFdlDfjnfsl7hTfg6zFtveiYEsNhNOuMVuxV9C8QbbS5MwzH6u+8hEERpctyrdZpP2Yw7/NpgbPIK5QW8TH+s66uUXz8jkz+9t56Uv8rl9weEBen5FHa+v38dNswfhyNluE/aVUkr1C8nRYTx9/Swu/PsKrnpuB//67jmkTbq4e2/qbrFTkYVG2Za5zjRUwcF8qMyHmmIYNN42znSWjuLwBI+t04llTLO5qWCnHTuwywaa5Ts9Xee5sGOZbWX35h14xw+BuME2eInLPjx4aW6wXxhqS+2iIuFxdo7i3gpUjbEzZuz6xH6hKFpv0wM4xoQ0QSH2C0VYtG3hDI2yUze2biERdkGiw4538jjE65zgsCP/OzQ32FSe4o2w/0vP/0muDarbpzCEJ8Bpd8HMm7qe+jGQpE6wWx+kwXWrsm12laB233rWNA1hLhBTsQnij5yjkx4Xzkkjk3lpTQG3nD4Kh9fgx0c+ySNIYHHmXljntlPhKKWU6jfs6rwzuezhlVzz+Oe89K15xEZ0fXXeDoIcNtXjaJwx4PRBjmxQ0KFUlPYaazy5qp5W9dbAe/u7Nng+7H1CbEt8o2eAWHuD58Opv7Bd8j2lap9NEch5Dso8a1XEZduF4CZfbtNhWgOwugqorzi0rz9g699Y7bVV2eNVhdBUB821du+q73qZxGFThBwhNsXHEWYfAxzYfWhGjpBI+3878nSISLI9HBEJdh+eYMsdGumzfyrVczS4blW6DYZ0HLT4flUa3wfbvTb8lKO+xaUzs/jus2v5eHspp4yxgXhpdSMvrM7ngikZJKy9zbYCDDvZ58VXSinlXxMyYnn46ulc+/hqLn9kJZOzYmluMbha3LjcBleLIS3OybmT05maFdc/VnkMi7K52GmTOr7WVGdzXCv3QuUe24peXWxTUyKT7RY1CCIH2XzwT/4HnlhoU1ZOvfP4B+67W6Bki02NKfjCHmtN5WhN7Wiut+kueR/YWS6y5sC5f4Ex50JkYufvG5N+fOUB2/rfXGtTgZpqbQpHY40nNcj7uOc1V6NN+3E12jz8lkZbr/Hn20VLUifZ/OA+ksagfEuDa7Bdb1WFdtlz78PNLawvEypjMojbt+6Yb3P62BQSIkN5YXV+W3D95PJdNLW4uWV4ISxdA+c8cOgbrFJKqX5l3vAk/nr5VO59YxP/3VJCSJAQ7AgiOEhwBAnvbyvhic92k5UQzqLJ6Zw3JYNRKcdeiKauycXmfVVs2lfFhIxYpg+OP+Y1fhcaYf8utvvb2KnM6XbA2epH4NMH4JFTYMw5MO58G9S2bt7zJLcO/qwuhuoiG1DvXWnnZG5tGY9Mti2/9RU2r91bbBaceLttofbnan5gg+CwaLspdQwaXIMdXAKHT8MH7Nhfg9tAfdJE4vblHPNtQoODuGBqBk8u301ZTSNhwUE8vWIPX5+QRtqGu+xSoVOu9EcNlFJK9ZCFE1JZOCG109eqG5p5Z9N+Xs8p5O8f5vLgB7mMSY1m+KAoosOCiQoLJspp98bA5qIqNhYeJLe0htbJDUMdQTx1/SzmDj9CC2xfFRoB838I06+zi4Cs+D+7sp+3iCSbDlPnmf72MAKDxsKECyF7js3hjh9yKH+5ueFQWofbZVt/teVX9UEaXIMdXAAdgustnmXPQ7OnwYq37S/0Mdaev3RmFo9+uotX1xbichuqG1zcPqYC3vgEzvyd/1Y3Ukop1euinSFcND2Ti6ZnUlrdyFsbi3j7y2K2FFVR0+CiptFFXdOhVfBSYsKYmBHL2RPTmJgRy5CkCL79z7UsfmYNL397HqNT+2FLqTMGTv4pzP+BHYxZVejZ9tl9XTlEzLUNTtGph/Zx2UefDSPECSHp3UvvUKoHaHANNrh2hNp8aC9bi6pxhgQRN3y2XYusKMdOfXQUI1OimZodx5LP91LV4OLEkUkM23K/HaAw/Ro/VkIppVRfkhwdxjXzhnDNvCGHHXe1uKltasHtNsRHhna47qnrZ/GNBz/j2ic+55XvziMtNryHSuxjIeFdTytRKoBofwrYEcaJIztMLL61uIrRqTE4MibbA11IDQG7YmNeWS1lNY38eGID7FwGc76ro36VUkoR7AgiNjyk08AaICMunCevm0V1g4trH1/NwfrmTs9TSvVNGlyDbblut3iMMYYtRVWMTY22I5Tjh9qVkrrgnMnpRIQ6mJodx8Rdj9pVlWbd5I+SK6WUCkDj0mP4x1XTyS2t4VvPrKHR1dLpeW73MeZxVkr1OE0Laa6HA3vsaGMvJdWNHKhrZkxrvlv6FDvVUBdEhQXz7I2zSW3cjTy71C5X6oz1dcmVUkoFsBNGJnH/xZO49YX13P7iei6clkluaQ25pTXsLKkht7SW2kYXc4cncuqYQZwyehBZCYevEHygtolVu8pZkVtObmkt3z15OPNGHGN+baVUt2hwXbYDMB1arrcU2cGMY1uXPU+bApte7dKgRoCp2fHwyk/tqk2zv+PrUiullBoAvjE1k+KDjfzh7a28uaEIgMTIUIYnR3Hm+BRCHEF8tL2Uu17fBGxi5KAoThkziBa3YXluOVuLqzAGwkMcRDmDufbJ1Tx01fS26WI70+Ry88gneYxPj+Hk0Uc+TynVOQ2uS7fZffuZQoqqARiT6gmu06fa/a6Pu7Z8eUWeneB+znePPKG9UkopdQzf/towpmXH4QgShidHdZqrnVdawwfbSvlgawlPfLYLEWHG4HhuO30Uc4cnMikzjtpGF1c//jmLn1nDXy+bylkT0zq8T35FHTc/t5b1BQcJCw7ixW/NZXLWUWbwUEp1oMF16Va7VGnC4RPQby2uIj3WeWh526zZdtDj23fA0JOO3Xr96QN2qdh53/dTwZVSSg0EIsLsYUdvpBmWHMWw5ChuOGEo9U0tiIAzxHHYOaHBoTx702yue2I1Ny9Zxx9dbs6fmtH2+rubivnRS+sxwH0XTuIv7+1g8TNrWHrzCaTEOP1RNaUCkl8HNIrIQhHZJiI7ReRnnbx+rYiUikiOZ7vR67UWr+NL/VbIsm2QMAyCD28J2FpUfSglBOz8mhc+ArUl8OYttM3235kv/wVrn4YZ19u5O5VSSqkeEh7q6BBYt4pxhvD09bOYNSSBW1/M4fnP99LkcvOrNzez+JkvGJwYyb+/fyKXzMzi0WtmUN3gYvEzX9DQ3PmASqVUR34LrkXEATwInAWMAy4XkXGdnPqCMWaKZ3vU63i91/FF/ionpds65Fs3ulrILa1hTFq7yfvTp8Kpd8Lm1yHnuc7fb+8qePU7kD0XzrjHT4VWSimljk9kWDBPXDeTk0Ym87NXNrLwzx/z2Ke7uGbuYF7+zlyyE+2gyLFpMfzpkimsz6/kZ//agDlao5JSqo0/W65nATuNMXnGmCbgeeA8P/68r87VBOW5HYLrnSU1uNzmUL61t3k/gMEnwH9+YvOqvVXkwfOXQ2wmXPacrsaolOpXutnbeI2I7PBsumJWH+cMcfDw1dM5a0IqpdWNPHjFNO45bwJhwYe3eC+ckMrtZ4zitZx9/OOjvCO82+EaXS2szCvnbx/uZENBpT+Kr1Sf5s+c6wwg3+t5ATC7k/MuFJGTgO3ArcaY1mucIrIGcAG/N8a81v5CEVkMLAbIzs7+6iWsyAPT0mEw41bPYMbD0kJaBTnggofg7/PglcVw3dt28Zm6Cnj2YjBuuPKlLs0oopRSfYVXb+MZ2Pv1ahFZaozZ3O7UF4wxN7e7NgG4G5gBGOALz7UHeqDo6jiFBTv425XTaGpxdwiqvd186gi27a/mvne2MnJQFKePS2l7rbnFTU2Di7yyWlbmlbM8t4w1uw/Q6HID8OfgHfzx4smcO1mXLFcDR28PaHwDWGKMaRSRbwFPAa3riw82xhSKyDDgfRHZaIzJ9b7YGPMw8DDAjBkzvnp/VelWu086fGnWLUVVhAUHMSQxopOLsC3T5zwAL18PH98PJ94GL1wFlXvh6qWQOLzz65RSqu9q620EEJHW3sb2wXVnzgSWGWMqPNcuAxYCS/xUVuUjInLUwLr1nPsvmsye8jpuXrKWtNhwqhtc1DQ209DsPuzcManRXDE7m3nDkxidEs3tL+Xw/SXrKDhQz7e/NgwR8Wd1lOoT/BlcFwJZXs8zPcfaGGPKvZ4+Ctzn9VqhZ58nIh8CU4HDgutuK90GSIfgemtxNaNSogl2HCVrZsKFsGMZfHwf7F0Bez6DCx+DwXN9WkSllOoh3elt7OzajE6u7X6Po+oV4aE2jeT+t7fR7DZEhQUT4wwmKiyYKGcwqTFOZg1NIDHq8HTIZ26YzY9f3sAf3t5KwYE67lk0/uh/W5UKAP4MrlcDI0VkKDaovgy4wvsEEUkzxhR5ni4CtniOxwN1nhbtJGA+XoG3z8z9How+C0IPtVC7Wtzk5FeyaEoXurDOug/2LIddH8Epd8LEi3xeRKWU6kOO1tvYJd3ucVS9Ji02nD9dOuUrXeMMcfCXS6eQGR/O3z/MpbCynv+7YhpRYTb8cLsNFXVNFB9sIMYZ0jaYUqn+zG/BtTHGJSI3A+8ADuBxY8wmEbkXWGOMWQr8QEQWYfOqK4BrPZePBR4SETd20OXvO8n7676wKEibdNihjYUHqWl0MW94FxZ+ccbY/Oo9y2H6tT4vnlJK9aDu9DYWAie3u/ZDn5dQ9UtBQcJPF44hKz6CX7z+Jef+76ckRIZSfLCBkuoGmlsOfceaOSSeS2dm8/WJqUSE+q/9L6+0hi/2HGBsWgxjUo/RU63UV+TXnGtjzFvAW+2O3eX1+A7gjk6uWw5M9GfZjmR5rv3bMfcYE/a3SR7dYbYRpZTqh467txHbiPJbT68jwAI6uberge2K2dmkxzn507LthDiEmUPiSYl1khpjt93ldby4Jp8fvbSeXy7dxKIp6Vw6I4tJmbE+ydV2uw0f7Sjlyc9289H20rbj4SEOJmXGMm1wPNOy45k7PLGtZV2p46GfnnZW5JYzJjW6Q96YUkoFsu70NhpjKkTkV9gAHeDe1sGNSnk7efQgTh496Iivf/trw1i9+wDPr97LK2sLeG7VXkKDg0iLdZIW6yQ9Npy0OCdZ8RFMzIxl9LHGRwHVDc28/EUBT6/Yw66yWgZFh3HbGaNYMD6F7ftrWLvnAGv3HuCRj/NwuQ3ZCREsWTyHjLhwX1dfDRASKJPCz5gxw6xZs6Zb79HQ3MLke97lytmDuevczta7UUop/xCRL4wxM3q7HD3JF/dtFbiqGpp5e2MxuaU17DvYQFFlPUUHGyiuaqDFbWOX1lbnqdnxTM2OI9oZzK6yWnaV1tp9WS17K+pwuQ1Ts+O4dt4QzpqQRmhwx4C8vqmF5bll3PJ8DnGRISy5aQ6Z8ZoDrjp3tHu2tlx7Wbe3kkaXu2v51koppZTymxhnCJfMzOpwvMVtKDhQR05+Jev2VrIuv5LHPs07LHc7LDiIoUmRjE6NZuGEVM4cn8rkrLij/rzwUAenjU3hmRtn883HVnHZwytZctMcshI0wFZfjQbXXlbklhEkMGuYLgCjlFJK9UWOIGFwYiSDEyM5b4qd8bGhuYVN+6poaG5haFIkqTFOgoKOL097SlYcz944m6setQH284s1wFZfjQ6P9bI8t5yJmXHEOEN6uyhKKaWU6iJniIPpg+OZPyKJ9Ljw4w6sW03KjOO5m+ZQ0+jisodXkl9Rd9zv1dziJr+iji/2HKCm0dWtcqn+QVuuPWobXeTkV3LTScN6uyhKKaWU6mUTMmJtC/Zjq7j0oRV862vDmZARw9i0mA7TBBpjKKluZNO+g2zeV0VeWS0FB+opPFBP0cF6PCniBAcJ0wbH87VRyZw4MokJ6bHd/iKg+h4Nrj1W767A5TbMH57U20VRSimlVB8wISOW526cw+Jn1nD30k0ABAkMS45iQnoMydFhbC2uZktRFWU1TW3XpcY4yUoIZ9bQBLLiw8mMjyAuIoR1+ZV8vL2U+9/Zxv3vbCM+IoQTRyZz+rgUTh6drD3nAUKDa4/lueWEOoKYPjj+2CcrpZRSakAYlx7DJz85hf1VjXxZeJCNhQfZtO8gK/MqKK9tZOSgaE4ZPYhx6TGMT49lbFo00UcIkheMT+WnC8dQWt3IZzvL+Hh7KR9uL2Xp+n2EOIQ5wxI5Y1wKp49NId3HUwHWNrrILa2hvKaJOcMSCQ91+PT91SEaXHsszy1janacftiUUkopdRgRITXWSWqsk9PHpbQdd7vNcaV1JEeHcf7UDM6fmkGL27B27wH+u3k/yzbv567XN3HX65sI80wX6D1hcognreTEkUmcNCqZ0SnRHRbYqW10sbW4ii1F1ewsqSG3tIadJTUUHWxoOycuIoRLZ2Rx1ZzBOljTDzS4Birrmti0r4pbThvV20VRSimlVD/hi3xpR5Awc0gCM4ckcMfXx7KzpIb3t+6nvPZQmolgf05to4uVeeX89q2t/PatrQyKDuPEkclkJYSzzZOesrv80ODLyFAHwwdFMWdYIiMGRTE8ORJniIMX1+Tz6Ke7ePiTPE4bk8K184Ywf0TiMVfCPFjfzB/f3ca/NxSx+KRhXDd/aKdzhg90GlwDK/MqMAbmjdD5rZVSSinVe0YMimLEoKijnlN0sJ5Ptpfx0Y5S3tu6n8q6ZoYkRjA2LYYLpmUyNi2GsWnRZMSFdxownzx6EPsq63l21R6WfJ7Pf7fsZ8SgKK6eO5gLpmV2WP7dGMPrOfv49b+3UFHbyISMWH73n628uCafe8+bwPwROl7Nm67QCNz9+pe8uKaA9Xcv0G9gSqleoSs0KqWOR4vb0ORyH3daa0NzC29uKOLpFbvZUHCQ6LBgLpyeyTXzhjA0KZKdJdXc+dqXrMyrYHJWHL85fwITMmJ5b8t+7nljM3sr6jhnUhp3nj2O1FinbyvXh+kKjcewPLecmUMTNLBWSimlVL/iCJJujRdzhji4aHomF07LYF1+JU8t382zq/bw5PLdzBgcz/qCSsJDHPzmGxO4bGY2Dk8qzGljU5g/IomHPsrjbx/u5P2tJZw/NYPUGCcJkaEkRYWSEBlGUlQoQxIjjzuFpqqhmS8LDzI1K77fjIsb8MF1SVUDO0pquGh6Zm8XRSmllFKqV4gI07LjmZYdz8/PHsuSVfm8vr6Q86Zk8LOzxpAUFdbhGmeIgx+ePpJvTM3gN29t5s31+6hq6LhQzvDkSK6bP5QLpmV0mCO8MyVVDSzbsp93Nu1nRW4ZzS2GcWkxPHLNDDJ8PIuKPwz4tJDXcwr54fM5vHHzCUzMjPVDyZRS6tg0LUQpFQiaXG4O1DVRVtNIRW0T+RX1PL96LxvmApx1AAALsElEQVQKDhLjDOby2dlcPXdIW5Dc0NzSNqPJzpIaPt1Zxrq9lQAMSYzgzPGpDE2K5Df/3kJYSBD/uGo6M4Yk9GYVAU0LOarlO8uJcQYzLj2mt4uilFJKKdWvhQYHkRLjJCXmUP715bOyWLv3AI9/uptHP9nFo5/sYlp2HMVVDRQcqKe1nTdI7Lzit58xigXjUxmVEtU2IHPGkARufGo1lz+ykt98YyKXzMg67OfWNLp4Y/0+Xl1bSHJ0GLeeMeqYA0MbmlvYWlzNlKw4n/4b+DW4FpGFwF8AB/CoMeb37V6/FrgfKPQc+j9jzKOe164B7vQc/7Ux5il/lHF5XhlzhiW25RAppZRSSinfERGmD05g+uAECivreXrFblbmljM5M44Lp2W2zZAyJNFOFdiZEYOieO1787n5uXX85OUNbC+u5mdnjWF9wUFeWL2XNzcUUdfUwohBUWwuquLtTcVcMiOLW04feVigD5BfUcezq/by4pp8GptbWPXz0zvMkNIdfguuRcQBPAicARQAq0VkqTFmc7tTXzDG3Nzu2gTgbmAGdv70LzzXHvBlGfMr6sivqOeG+UN9+bZKKaWUUqoTGXHh3HHW2OO6Ni4ilCevm8mv/72FRz/dxb/WFnCgrpmIUAeLJqdz6cwspmTFUVHbxP++v5NnV+3h1XUF3HjCMG46aRhr9x7gnyv28P62EgQ4Y1wK35wzhIgjBPTHy58t17OAncaYPAAReR44D2gfXHfmTGCZMabCc+0yYCGwxJcFXJ5bBsA8nZ9RKaW60tt4G3Aj4AJKgeuNMXs8r7UAGz2n7jXGLOqxgiulBoxgRxC/XDSecWkxvLOpmAXjUzh7UvphLc+JUWH8ctF4rp8/lP95dxv/98FO/v5RLi1uQ1JUGDefMoLLZ2X7fIn5tjL65V2tDCDf63kBMLuT8y4UkZOA7cCtxpj8I1yb0f5CEVkMLAbIzs7+ygWMiwhlwbgURh4jJ0cppQJdF3sb1wEzjDF1IvId4D7gUs9r9caYKT1aaKXUgHXJzCwumZl11HOyEyP46+VTWXzSMF5ak8+MIQmcOT7V71Mv9/aAxjeAJcaYRhH5FvAUcGpXLzbGPAw8DHbU+Vf94WeOT+XM8alf9TKllApEx+xtNMZ84HX+SuCqHi2hUkodhwkZsUzI6LkZ4fwZuhcC3l8pMjk0cBEAY0y5MabR8/RRYHpXr1VKKeVTXeox9HID8B+v504RWSMiK0Xk/CNdJCKLPeetKS0t7V6JlVKqD/JncL0aGCkiQ0UkFLgMWOp9goikeT1dBGzxPH4HWCAi8SISDyzwHFNKKdXLROQq7IDz+70OD/bM+XoF8GcRGd7ZtcaYh40xM4wxM5KTk3ugtEop1bP8lhZijHGJyM3YoNgBPG6M2SQi9wJrjDFLgR+IyCLs4JgK4FrPtRUi8itsgA5wb+vgRqWUUn7RpR5DETkd+DnwNa+eR4wxhZ59noh8CEwFcv1ZYKWU6ov8mnNtjHkLeKvdsbu8Ht8B3HGEax8HHvdn+ZRSSrVp623EBtWXYVuh24jIVOAhYKExpsTreDxQ5xk/kwTMxw52VEqpAae3BzQqpZTqA7rY23g/EAW85Fk1rXXKvbHAQyLixqYb/r6TNQ2UUmpA0OBaKaUU0KXextOPcN1yYKJ/S6eUUv2Dfyf6U0oppZRSagDR4FoppZRSSikfEWO+8torfZKIlAJ7juPSJKDMx8XpawK9joFePwj8OgZ6/eDYdRxsjBlQc9PpffuItH79X6DXMdDrB924ZwdMcH28RGSNZ27WgBXodQz0+kHg1zHQ6wcDo449JdD/LbV+/V+g1zHQ6wfdq6OmhSillFJKKeUjGlwrpZRSSinlIxpcw8O9XYAeEOh1DPT6QeDXMdDrBwOjjj0l0P8ttX79X6DXMdDrB92o44DPuVZKKaWUUspXtOVaKaWUUkopH9HgWimllFJKKR8Z0MG1iCwUkW0islNEftbb5fEFEXlcREpE5EuvYwkiskxEdnj28b1Zxu4QkSwR+UBENovIJhH5oed4QNRRRJwi8rmIrPfU7x7P8aEissrzWX1BREJ7u6zdISIOEVknIm96ngda/XaLyEYRyRGRNZ5jAfEZ7U16z+5/9J4dGPc0COz7tq/v2QM2uBYRB/AgcBYwDrhcRMb1bql84klgYbtjPwPeM8aMBN7zPO+vXMDtxphxwBzge57/t0CpYyNwqjFmMjAFWCgic4A/AA8YY0YAB4AberGMvvBDYIvX80CrH8ApxpgpXvOkBspntFfoPbvf0nt24NzTAv2+7bN79oANroFZwE5jTJ4xpgl4Hjivl8vUbcaYj4GKdofPA57yPH4KOL9HC+VDxpgiY8xaz+Nq7C96BgFSR2PVeJ6GeDYDnAq87Dneb+sHICKZwNnAo57nQgDV7ygC4jPai/Se3Q/pPRvox/VrNUDv28f9GR3IwXUGkO/1vMBzLBClGGOKPI+LgZTeLIyviMgQYCqwigCqo6frLQcoAZYBuUClMcblOaW/f1b/DPwEcHueJxJY9QP7x/VdEflCRBZ7jgXMZ7SX6D27n9N7dr8W6Pdtn96zg31dOtW3GWOMiPT7+RdFJAr4F3CLMabKfom2+nsdjTEtwBQRiQNeBcb0cpF8RkTOAUqMMV+IyMm9XR4/OsEYUygig4BlIrLV+8X+/hlVPSdQPit6z+6/Bsh926f37IHccl0IZHk9z/QcC0T7RSQNwLMv6eXydIuIhGBv0s8aY17xHA6oOgIYYyqBD4C5QJyItH4Z7s+f1fnAIhHZje3WPxX4C4FTPwCMMYWefQn2j+0sAvAz2sP0nt1P6T27339WA/6+7et79kAOrlcDIz2jXUOBy4ClvVwmf1kKXON5fA3wei+WpVs8eV6PAVuMMX/yeikg6igiyZ7WD0QkHDgDm6P4AXCR57R+Wz9jzB3GmExjzBDs79z7xpgrCZD6AYhIpIhEtz4GFgBfEiCf0V6k9+x+SO/ZQD+uHwT+fdsf9+wBvUKjiHwdm0fkAB43xvyml4vUbSKyBDgZSAL2A3cDrwEvAtnAHuASY0z7ATT9goicAHwCbORQ7tf/w+bw9fs6isgk7MAJB/bL74vGmHtFZBi2xSABWAdcZYxp7L2Sdp+ne/FHxphzAql+nrq86nkaDDxnjPmNiCQSAJ/R3qT37P5H79n9/57mLRDv2/64Zw/o4FoppZRSSilfGshpIUoppZRSSvmUBtdKKaWUUkr5iAbXSimllFJK+YgG10oppZRSSvmIBtdKKaWUUkr5iAbXSvmIiJwsIm/2djmUUkodm96zlb9ocK2UUkoppZSPaHCtBhwRuUpEPheRHBF5SEQcIlIjIg+IyCYReU9Ekj3nThGRlSKyQUReFZF4z/ERIvJfEVkvImtFZLjn7aNE5GUR2Soiz3pWJ1NKKXWc9J6t+hsNrtWAIiJjgUuB+caYKUALcCUQCawxxowHPsKukgbwNPBTY8wk7ApjrcefBR40xkwG5gFFnuNTgVuAccAwYL7fK6WUUgFK79mqPwru7QIo1cNOA6YDqz0NFOFACXZZ3hc85/wTeEVEYoE4Y8xHnuNPAS+JSDSQYYx5FcAY0wDgeb/PjTEFnuc5wBDgU/9XSymlApLes1W/o8G1GmgEeMoYc8dhB0V+0e48c5zv3+j1uAX9HVNKqe7Qe7bqdzQtRA007wEXicggABFJEJHB2N+FizznXAF8aow5CBwQkRM9x78JfGSMqQYKROR8z3uEiUhEj9ZCKaUGBr1nq35Hv6GpAcUYs1lE7gTeFZEgoBn4HlALzPK8VoLN8QO4BviH50acB1znOf5N4CERudfzHhf3YDWUUmpA0Hu26o/EmOPtSVEqcIhIjTEmqrfLoZRS6tj0nq36Mk0LUUoppZRSyke05VoppZRSSikf0ZZrpZRSSimlfESDa6WUUkoppXxEg2ullFJKKaV8RINrpZRSSimlfESDa6WUUkoppXzk/wOU7dsksYr6tAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_history(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the trained model on test citation links:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Set Metrics of the trained model:\n",
      "\tloss: 0.2011\n",
      "\tacc: 0.9495\n",
      "\n",
      "Test Set Metrics of the trained model:\n",
      "\tloss: 0.7457\n",
      "\tacc: 0.7429\n"
     ]
    }
   ],
   "source": [
    "train_metrics = model.evaluate_generator(train_flow)\n",
    "test_metrics = model.evaluate_generator(test_flow)\n",
    "\n",
    "print(\"\\nTrain Set Metrics of the trained model:\")\n",
    "for name, val in zip(model.metrics_names, train_metrics):\n",
    "    print(\"\\t{}: {:0.4f}\".format(name, val))\n",
    "\n",
    "print(\"\\nTest Set Metrics of the trained model:\")\n",
    "for name, val in zip(model.metrics_names, test_metrics):\n",
    "    print(\"\\t{}: {:0.4f}\".format(name, val))"
   ]
  }
 ],
 "metadata": {
  "file_extension": ".py",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "mimetype": "text/x-python",
  "name": "python",
  "npconvert_exporter": "python",
  "pygments_lexer": "ipython3",
  "version": 3
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
